Full-Stack & Applied AI

PixiEstim

An intelligent software project estimation platform that transforms heterogeneous project documents into structured, AI-assisted software estimations through controlled LLM workflows, historical RAG context, deterministic business rules, and human validation.

Role

Software Engineer — Full-Stack & Applied AI

Technologies

Next.js React TypeScript Tailwind CSS NestJS FastAPI Python PostgreSQL Prisma Redis BullMQ MinIO Qdrant Ollama OpenDataLoader RAG Prompt Engineering Human-in-the-Loop Langfuse Docker Docker Compose Jenkins CI/CD

The problem

Software project estimation at PixiMind relied heavily on manually reading heterogeneous requirements documents, identifying functional requirements, selecting the appropriate estimation scales, evaluating implementation complexity, and consolidating effort estimates. The process could become time-consuming, difficult to standardize, and strongly dependent on the experience of the people involved, especially when documents were large, incomplete, or ambiguous.

The solution

PixiEstim centralizes the software estimation workflow in a single platform. Project documents are uploaded and processed asynchronously, converted into structured and context-preserving segments, and analyzed through specialized AI workflows. The system proposes software features, associates validated features with internal estimation scales, evaluates their complexity, enriches decisions with similar historical cases through RAG, converts complexity levels into deterministic effort values, and keeps Sales and Production users in control through explicit human validation stages.

System Architecture

PixiEstim follows a distributed architecture composed of a Next.js frontend, a NestJS backend responsible for business logic, authorization, workflow coordination, and persistence, and a dedicated FastAPI service responsible for document processing and AI orchestration. PostgreSQL and Prisma manage structured business data. Redis and BullMQ support asynchronous processing, MinIO stores uploaded project documents, Qdrant stores and retrieves historical estimation knowledge through vector search, and Ollama provides local LLM inference. The application components are containerized with Docker and integrated into Jenkins-based CI/CD workflows.

Applied AI

AI Engineering

PixiEstim implements a controlled AI workflow rather than delegating the full estimation process to a language model. Each intelligent stage has a specific responsibility, limited context, specialized prompt, expected structured output, application-level validation, fallback rules, traceability, and human validation where required.

AI Pipeline

The workflow is decomposed into specialized and independently controlled stages instead of relying on a single end-to-end prompt.

Document ingestion

Project documents are uploaded, stored in MinIO, and submitted to an asynchronous processing workflow instead of being processed directly inside the request lifecycle.

Document extraction

FastAPI retrieves the document, detects its format, selects the appropriate extractor, and converts PDF, DOCX, or TXT content into a common internal representation.

Normalization and reconstruction

Formatting noise and redundant visual information are removed while preserving section hierarchy, reading order, titles, paragraphs, lists, tables, and source provenance.

Contextual segmentation

Large documents are divided along natural document boundaries into context-preserving segments compatible with the local model context window.

Project synthesis

A specialized LLM call creates a global project context from the project name, main document sections, and relevant metadata before detailed feature extraction begins.

Progressive feature extraction

Document segments are processed progressively in batches. Each batch receives the global project context and previously extracted feature titles to preserve coherence and reduce duplication.

Structured output validation

Generated outputs are parsed as JSON and validated against expected fields, formatting rules, closed candidate lists, and application-level constraints before persistence.

Feature deduplication

Generated feature titles are normalized and compared using a configurable similarity mechanism so strongly similar proposals can be consolidated before validation.

Human feature validation

Generated features remain proposals. Authorized users can modify, remove, reorder, correct, or validate them, and only validated features can proceed to estimation.

Estimation-scale association

Validated features are associated with one or more estimation scales selected for the current project. The model can only choose from the closed list of configured scales.

Historical RAG retrieval

For each feature-scale pair, PixiEstim retrieves semantically similar historical estimation cases from Qdrant while applying metadata filtering and relevance controls.

Complexity classification

The model evaluates each feature-scale pair using the current requirement, estimation scale, allowed complexity levels, and retrieved historical references.

Deterministic effort calculation

The LLM does not generate arbitrary effort values. It selects a complexity level, and PixiEstim converts that level into effort using internally configured estimation values.

Human estimation validation

Production users review scale associations, complexity levels, calculated effort, fallback cases, and unassigned features before validating the final estimation.

Prompt Engineering

Task-specific prompts with constrained context, closed candidate sets, explicit rules, and structured outputs.

Project synthesis

Build a global project understanding before detailed analysis.

Output

Structured project summary

Feature extraction

Identify functional requirements from document segments.

Output

Structured JSON list of software features

Scale association

Determine which configured technical estimation scales apply to each validated feature.

Output

JSON associations restricted to authorized project scales

Complexity evaluation

Classify each feature-scale pair according to the allowed complexity scale.

Output

One value from the predefined complexity levels

Explicit model role for each AI task Task-specific and limited context Structured JSON output requirements Closed lists of authorized estimation scales Closed complexity scale Explicit forbidden outputs Application-level schema validation Previously extracted feature titles included during progressive extraction

RAG Engineering

Ground complexity evaluation with comparable historical software estimation cases without treating historical decisions as automatic answers.

Vector DB

Qdrant

Top K

Similarity threshold

≥ 0.55

Context

Historical cases

Embedded context

→ Feature name
→ Estimation scale
→ Associated service
→ Concerned part

Stored metadata

→ Historical complexity
→ Reference identifier
→ Estimation scale
→ Associated service
→ Concerned part
→ Embedding/index version

Key RAG decisions

✓ Historical complexity is stored as metadata but excluded from the semantic text used to build the search vector.
✓ The same embedding configuration and preprocessing are used for indexing and retrieval.
✓ References are filtered according to the relevant estimation scale before being added to the prompt.
✓ Only references meeting the configured similarity threshold are retained.
✓ Historical references are presented to the model as comparison cases rather than answers to reproduce.

Human-in-the-Loop

Human validation is used as a control mechanism at two critical stages of the AI workflow.

Feature validation

Production or authorized users can edit, remove, reorder, deduplicate, and validate generated functional requirements. Only validated features enter the estimation pipeline.

Estimation validation

Users review associated scales, complexity classifications, calculated effort, fallback values, and unassigned features before approving the final estimation.

AI Reliability & Guardrails

Model outputs and retrieved context are validated before they can influence the business workflow.

Risk

Unparsable model response

Control

JSON syntax validation

Behavior

The response is rejected or retried before persistence.

Risk

Incomplete generated element

Control

Schema and required-field validation

Behavior

An invalid element can be discarded while keeping the remaining usable output.

Risk

Duplicate generated features

Control

Normalization and similarity comparison

Behavior

Strongly similar proposals are consolidated before human validation.

Risk

Unknown estimation scale

Control

Closed-list validation against project scales

Behavior

The association is rejected and the affected feature is flagged.

Risk

Invalid or missing complexity

Control

Validation against the predefined five-level complexity scale

Behavior

A technical fallback is applied and the result is explicitly marked for human review.

Risk

Out-of-domain historical reference

Control

Metadata filtering using the estimation scale

Behavior

The reference is excluded before augmented-context construction.

Risk

Low-relevance historical reference

Control

Configurable similarity threshold

Behavior

The reference is not injected into the prompt and its absence remains traceable.

Risk

Excessive RAG context

Control

Top-K result limitation

Behavior

Only the highest-ranked historical references are retained.

Risk

No reliable historical reference

Control

Validation of retrieved-result count

Behavior

The estimation continues without historical context and this execution mode is reported.

Risk

Blind reproduction of a historical estimate

Control

Explicit prompt instruction

Behavior

The model must reason from the current requirement and use historical cases only as comparison evidence.

LLM Observability

Instrumented around the FastAPI orchestration layer.

Langfuse

Project and execution identifier Model Prompt version Constructed input Raw model response Validated application result Execution duration Token consumption when available Retrieved RAG references Retrieval similarity scores Parsing errors Validation errors Fallback usage

Support debugging, compare prompt variants, identify slow AI workflow stages, examine retrieval context, detect recurrent failures, and distinguish raw LLM output from data accepted by the application.

Evaluation

No invented metrics

Operational observability is implemented, while a formal offline retrieval-quality benchmark is not reported as a completed result.

Currently observable

✓ Execution duration
✓ Prompt version
✓ Model information
✓ Raw output versus validated output
✓ Retrieved references
✓ Retrieval similarity scores
✓ Parsing failures
✓ Validation failures
✓ Fallback usage
✓ Token consumption when available

Offline benchmark targets

→ Recall@K
→ MRR
→ RAG versus non-RAG estimation comparison
→ Exact complexity agreement with human validation
→ Within-one-level complexity agreement
→ Structured-output validity rate
→ Fallback rate
→ p50 inference latency
→ p95 inference latency

Technical challenges

→ Processing heterogeneous and potentially large project documents without blocking the main application.
→ Preserving semantic structure and source context while converting PDF, DOCX, and TXT documents into AI-ready representations.
→ Adapting large documents to the limited context window of a locally hosted language model.
→ Extracting coherent software features progressively without losing the global project context.
→ Preventing malformed, incomplete, duplicated, or out-of-domain LLM outputs from entering the business workflow.
→ Associating generated features only with estimation scales authorized for the current project.
→ Retrieving relevant historical examples without introducing unrelated or overly influential references.
→ Keeping AI-generated recommendations under human control throughout feature generation and estimation.
→ Maintaining traceability across document processing, LLM calls, RAG retrieval, validation, fallback, and persistence stages.
→ Coordinating frontend, backend, AI services, PostgreSQL, Redis, BullMQ, MinIO, Qdrant, Ollama, and observability components within one workflow.

Key engineering decisions

→ Separated AI and document-processing responsibilities into a dedicated FastAPI service while keeping business rules, authorization, and persistence inside NestJS.
→ Used Redis and BullMQ for asynchronous processing so document ingestion and AI workloads do not block the main application API.
→ Used MinIO object storage for uploaded project documents instead of storing binary files directly in PostgreSQL.
→ Used PostgreSQL and Prisma for structured business entities such as projects, users, clients, services, estimation scales, features, and estimation history.
→ Converted heterogeneous documents into a common internal representation before AI processing.
→ Used contextual segmentation based on natural document boundaries rather than fixed character-based chunking.
→ Used specialized prompts for project synthesis, feature extraction, scale association, and complexity evaluation instead of one general-purpose prompt.
→ Used structured JSON outputs and application-level validation around LLM responses.
→ Restricted estimation-scale associations and complexity classifications to closed lists of authorized values.
→ Used Qdrant to retrieve semantically similar historical estimation cases.
→ Excluded historical complexity from the embedded search text so the target value does not directly influence semantic similarity.
→ Applied scale-based filtering, Top-K retrieval, and a configurable similarity threshold before injecting historical context.
→ Converted AI-selected complexity levels into effort through deterministic business values instead of allowing the LLM to invent durations.
→ Introduced human validation between feature generation and estimation and again before final estimation approval.
→ Instrumented intelligent workflows through Langfuse to trace prompts, responses, retrieval context, durations, errors, and fallback behavior.
→ Used local inference through Ollama to improve control over confidentiality, external dependencies, and operating costs.
→ Containerized application services with Docker and automated build and deployment workflows through Jenkins.

Results

✓ Built an end-to-end workflow from project-document ingestion to validated software effort estimation.
✓ Implemented asynchronous document ingestion and processing across NestJS, Redis, BullMQ, MinIO, and FastAPI.
✓ Created a document-preparation pipeline covering extraction, normalization, structural reconstruction, contextual segmentation, persistence, and source traceability.
✓ Implemented specialized LLM workflows for project synthesis, software-feature extraction, estimation-scale association, and complexity classification.
✓ Added structured-output validation, deduplication, fallback handling, and business-rule enforcement around LLM responses.
✓ Introduced human-in-the-loop validation before estimation and before final result approval.
✓ Implemented deterministic conversion from AI-classified complexity to company-configured effort values.
✓ Integrated historical estimation knowledge through a Qdrant-based RAG pipeline.
✓ Implemented filtered semantic retrieval with Top-K = 2 and a configurable similarity threshold of 0.55.
✓ Added RAG guardrails for domain filtering, low-relevance references, excessive context, missing references, and invalid complexity values.
✓ Added observability for AI workflows through Langfuse, including model and prompt information, inputs, raw and validated outputs, durations, retrieval references, similarity scores, errors, and fallback behavior.
✓ Containerized the frontend, backend, AI service, and supporting infrastructure and integrated them into Jenkins-based CI/CD workflows.