The RAG Prompt: The Rules of Evidence

A RAG chain is only as reliable as the Instructions you give it. If your prompt is too loose, the model will ignore the "Context" and hallucinate using its own training data. We need to turn the LLM into a Judicial Assistant that only looks at the evidence provided.

1. The "Strict" System Prompt

SYSTEM: You are a professional researcher. Use the provided context to answer the user's question. 

RULES:
1. ONLY use the provided context. 
2. If the answer isn't in the context, say "I cannot find information on that in our records." 
3. Include the source name from the context metadata if available.
4. DO NOT make up URLs or facts.

CONTEXT:
{context}

USER QUESTION:
{question}

---

2. Handling the "Empty Context"

Sometimes your Vector Store search returns 0 results (e.g., the user asked about "Space" in a "Cooking" database).

• Your prompt should instruct the model to be Honest about the lack of information.
• Pro Tip: You can add a check in your Python code to see if docs is empty and bypass the LLM entirely to save money.

---

3. Visualizing the Filter

graph TD
    Data[Retrieved Context] --> P[System Prompt]
    World[AI's Internal Knowledge] -- BLOCKED --> P
    User[Human Question] --> P
    P --> Out[Answer strictly from Data]

---

4. Multi-Persona RAG

You can change the Tone without changing the data.

• Persona A (Legal): "Based on clause 4.2 of the documentation..."
• Persona B (Retail): "Great news! Our policy says you can return that within 30 days!" The Context stays the same; only the Template changes.

---

5. Engineering Tip: XML Tags

Latest research shows that models like Claude and GPT-4 handle context better when it is wrapped in XML Tags.

<context>
{context}
</context>
<question>
{question}
</question>

The tags act as "Concrete Walls" that help the model separate the instructions from the data.

---

Key Takeaways

• Grounding prevents the model from using external "Hallucinated" facts.
• Specific Rules (No URLs, No guesses) improve enterprise reliability.
• Metadata should be used for citation reporting.
• XML tags help models process long context sections accurately.