Generation and Verification Layer

The final layer generates responses and ensures accuracy through verification.

Generation Pipeline

graph LR
    A[Retrieved Docs] --> B[Assemble Context]
    B--> C[Add System Prompt]
    C --> D[LLM Generation]
    D --> E[Verify Response]
    E --> F{Grounded?}
    F -->|Yes| G[Return Response]
    F -->|No| H[Retry or Decline]

Context Assembly

def assemble_context(query, retrieved_docs):
    context_parts = []
    
    for i, doc in enumerate(retrieved_docs):
        context_parts.append(f"""
Document {i+1} (Source: {doc.metadata['source']}):
{doc.content}
---
        """)
    
    full_context = "\n".join(context_parts)
    
    prompt = f"""
Context:
{full_context}

Question: {query}

Based ONLY on the provided context, answer the question.
Include source citations.
    """
    
    return prompt

LLM Generation

response = claude.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=2000,
    temperature=0.0,  # Deterministic for RAG
    messages=[{
        "role": "user",
        "content": assembled_prompt
    }]
)

Verification Steps

1. Hallucination Detection

def verify_response(response, source_docs):
    # Check if facts appear in sources
    claims = extract_claims(response)
    
    for claim in claims:
        if not appears_in_sources(claim, source_docs):
            return False, f"Unverified claim: {claim}"
    
    return True, "All claims verified"

2. Source Attribution

{
    "answer": "The return policy allows 30 days for full refunds.",
    "sources": [
        {"document": "policy.pdf", "page": 3, "quote": "...30-day return..."}
    ],
    "confidence": 0.95
}

3. Answer Quality Checks

• Relevance to query
• Completeness
• Clarity
• Citation coverage

Final Response

{
    "question": "What is our return policy?",
    "answer": "Our return policy allows customers to return items within 30 days of purchase for a full refund. Items must be in original condition with tags attached.",
    "sources": [
        {
            "document": "customer_policy_2025.pdf",
            "section": "Returns and Refunds",
            "page": 3,
            "confidence": 0.98
        }
    ],
    "verified": true
}

Module 3 complete! Next module covers data types.