High-Level RAG Architecture

A production RAG system consists of multiple layers working together. This lesson explores the complete architecture.

The RAG Stack

graph TD
    A[Data Sources] --> B[Ingestion Layer]
    B --> C[Preprocessing Layer]
    C --> D[Embedding Layer]
    D --> E[Indexing Layer]
    E --> F[Vector Database]
    
    G[User Query] --> H[Query Processing]
    H --> I[Retrieval Layer]
    I --> F
    F --> J[Ranking Layer]
    J --> K[Context Assembly]
    K --> L[Generation Layer]
    L --> M[Verification Layer]
    M --> N[Response]

The Six Core Layers

1. Ingestion Layer

• Connects to data sources
• Handles batching and streaming
• Manages incremental updates

2. Preprocessing Layer

• Cleans and normalizes data
• Extracts text from images (OCR)
• Transcribes audio/video
• Parses documents

3. Embedding Layer

• Converts data to vectors
• Handles multimodal embeddings
• Manages embedding models

4. Indexing & Storage

• Stores vector embeddings
• Maintains metadata
• Optimizes for retrieval speed

5. Retrieval & Ranking

• Searches vector database
• Re-ranks results
• Filters by metadata

6. Generation & Verification

• Assembles context
• Generates responses
• Verifies accuracy
• Provides citations

Data Flow

sequenceDiagram
    participant User
    participant API
    participant Retrieval
    participant VectorDB
    participant LLM
    
    User->>API: "What is our return policy?"
    API->>Retrieval: Process query
    Retrieval->>VectorDB: Search embeddings
    VectorDB-->>Retrieval: Top 10 docs
    Retrieval->>Retrieval: Re-rank results
    Retrieval->>LLM: Top 3 docs + query
    LLM-->>API: Generated response
    API-->>User: Response + sources

Next lessons cover each layer in detail.