Evaluating Answer Accuracy

As a Power User, you cannot afford to take ChatGPT's word for it. You must develop a "Bullshit Detector" for AI outputs.

1. The "Confidence" Myth

Just because ChatGPT sounds confident doesn't mean it's right. The model is trained to be helpful and polite, which sometimes leads it to "fill in the gaps" with guesses.

2. Verification Strategies

The Search Engine Cross-Check

Always search for unique names, dates, or technical terms in a search engine. If you can't find them, they are likely hallucinated.

The "Reverse Prompt"

Ask the AI to prove its own answer.

• "You said that the CEO of [Company] is [Name]. Can you provide a link to their official LinkedIn or a news article that confirms this? If you can't find it, tell me."

graph TD
    Output[AI Output] --> BiasCheck[Check for Bias]
    Output --> FactCheck[Check Fact Accuracy]
    FactCheck -->|Search| Search[Search Engine Check]
    FactCheck -->|Reasoning| Logic[Logical Consistency Check]
    Search & Logic --> Final[Verified/Corrected Output]

3. Using the "Cite Sources" Instruction

Force the AI to be accountable.

• "Summarize the current state of 5G technology. For every claim you make, cite a specific industry report or news outlet. If you don't have a specific source, label the claim as 'General Knowledge'."

4. The Self-Correction Prompt

Ask a second instance of the AI (or the same one) to find errors in the first response.

• "Below is a response I received about [Topic]. Act as a fact-checker. Identify any potential inaccuracies or oversimplifications."

---

Hands-on: Spot the Error

1. Ask ChatGPT a question about a complex, obscure historical event.
2. Task: Find one fact in its response and try to find a second, independent source on the web that confirms or denies it.

Key Takeaways

• Confidence $\neq$ Accuracy.
• Use Reverse Prompts to test the model.
• Cite your logic to ensure the AI isn't skipping steps.