Module 9 Lesson 5: Reducing Hallucinations
Stick to the facts. Techniques to prevent local AI from making up information.
Hallucinations: Keeping AI Grounded
A "Hallucination" is when an AI gives an answer that sounds confident and logical but is factually wrong. In local models, this usually happens because the model is too small to contain the specific fact, but it "wants" to be helpful.
Here is how to fight it.
1. The "I Don't Know" Directive
By default, an AI will try to answer even if it's guessing. You must override this.
System Prompt: "If you do not know the answer based on the provided text, state 'I do not know'. Never use your own internal knowledge to fill in gaps."
2. Low Temperature (The Deterministic Dial)
As we learned in Module 5, higher temperatures allow the model to take risks.
- Accuracy Goal: Set
temperatureto0.0or0.1. - Why: This forces the model to always pick the most likely word. It reduces "creativity" but increases "fidelity."
3. The "Citation" requirement
Force the model to prove its work. "For every claim you make, quote the specific sentence from the document that supports it."
If the model can't find a quote, it is much less likely to make something up.
4. Setting the "Confidence" Guardrail
This is an advanced technique where you ask the model to rate its own answer.
- Prompt: "Answer the question. Then, provide a confidence score from 1-10."
- Filter: If your script sees a confidence score below 7, ignore the answer and ask for human help.
5. RAG (The Ultimate Fix)
The absolute best way to stop hallucinations is to not rely on the model's memory at all. In the next module (Module 10), we will learn RAG, which provides the model with "Search Results" or "Documents" to read before it answers.
A model with a "Source" is 10x less likely to hallucinate than a model relying on its training.
Key Takeaways
- Hallucination is a helpfulness-driven error.
- Use Temperature 0 to make the model more factual.
- Explicitly tell the model it is allowed to say "I don't know."
- Citations act as a check against "Confidence errors."
- RAG (External data) is the industry-standard solution for absolute accuracy.