Automatic License Plate Recognition

The problem

Vehicle identification systems need to transform visual information into structured vehicle records while also giving operators efficient access to contextual information such as violations, payments, and applicable traffic regulations.

The solution

The project combines an ALPR pipeline based on YOLOv8 and OCR with a FastAPI backend for vehicle dossiers and violations. It also integrates a RAG-powered legal assistant that retrieves traffic-law information and uses a Mistral language model to answer contextual legal questions. An agent mode can invoke application tools for more complex vehicle-related queries.

System Architecture

FastAPI exposes the ALPR, vehicle, chat, and agent APIs. Images are processed through a YOLOv8-based plate detection pipeline followed by OCR and plate normalization. MongoDB stores vehicle, violation, and payment data. Legal documents are indexed into a FAISS vector index for retrieval, while Mistral generates answers based on the retrieved context. Repository and service layers isolate persistence and business logic.

Technical challenges

• → Detecting license plates from images under varying image conditions.
• → Converting OCR output into normalized and usable license plate identifiers.
• → Connecting recognition results with persistent vehicle and violation records.
• → Retrieving relevant traffic-law passages for legal questions.
• → Combining vehicle context with RAG-based legal responses.
• → Exposing AI and computer-vision operations through clean backend APIs.
• → Supporting both conversational questions and tool-driven agent operations.

Key engineering decisions

• → Used YOLOv8 for license plate detection and segmentation.
• → Separated plate detection, OCR, and normalization into dedicated ALPR modules.
• → Used MongoDB for vehicle, violation, and payment documents.
• → Used FastAPI to expose recognition and AI functionality through REST endpoints.
• → Used FAISS to index and retrieve relevant traffic-law documents.
• → Integrated Mistral for language-model responses based on retrieved legal context.
• → Created a separate agent endpoint capable of using application tools for structured operations.
• → Separated API routes, repositories, services, schemas, RAG logic, and LLM integration into distinct modules.

Results

• ✓ Built a complete backend pipeline from uploaded vehicle image to recognized plate.
• ✓ Connected recognized plates with vehicle, violation, and payment information.
• ✓ Added a RAG-powered assistant for questions related to Tunisian traffic regulations.
• ✓ Implemented contextual legal queries using both retrieved documents and vehicle information.
• ✓ Added an agent mode for tool-based operations.
• ✓ Designed the backend so computer vision, persistence, RAG, and LLM components remain independently maintainable.