Module 11 Lesson 1: The Reasoning Gap (System 1 vs System 2)

As you use LLMs more, you will encounter a strange paradox: The model can write a beautiful Shakespearean sonnet about quantum physics, but it might fail to solve a middle-school math word problem.

This is because LLMs primarily operate using what psychologists call System 1 thinking, while true logic requires System 2 thinking. In this lesson, we explore this "Reasoning Gap."

---

1. System 1 vs. System 2

In the book Thinking, Fast and Slow, Daniel Kahneman describes two ways humans think:

• System 1 (Fast): Intuitive, automatic, and emotional. (e.g., Reading a facial expression or finishing the phrase "Salt and...").
• System 2 (Slow): Deliberate, logical, and computational. (e.g., Calculating 17 x 24 or filling out a tax form).

LLMs are the ultimate System 1 machines. They are built to "automatically" finish patterns based on statistical intuition. They do not have an internal "logic engine" that checks their work as they go.

---

2. Why "Thinking" in Tokens is Dangerous

Because the model predicts word-by-word, it can't "plan ahead." If it starts a sentence with a certain logical premise, it is statistically committed to that path, even if it realizes halfway through that the math is wrong.

Imagine you are trying to solve a maze, but you can only see 1 inch in front of you. You'll make great "local" turns, but you won't see the dead end coming from a mile away.

---

3. The Lack of a "World Model"

A human knows that if you tip over a glass of water, the water will spill because of gravity and liquid physics. A human has a World Model.

An LLM knows that the word "glass" is statistically likely to be followed by "water" and "spill." But it doesn't "know" the physics. If you describe a glass of water on the moon, or a glass made of glue, the LLM might still say the water "spills to the floor" because that is the most likely text pattern in its training data.

graph TD
    subgraph "Human Thinking"
        Fact["Observation"] --> Logic["Internal World Model (Logic/Physics)"]
        Logic --> Conclusion["Accurate Result"]
    end
    subgraph "LLM Thinking"
        Token["Input Tokens"] --> Pattern["Statistical Pattern Match"]
        Pattern --> Prediction["Next Probable Token"]
    end

---

4. Why CoT (Chain of Thought) helps

This is why "System 2" techniques like Chain of Thought (Module 10) are so important. By forcing the model to write its steps out, we are effectively tricking a System 1 machine into acting like a System 2 machine. Each written step becomes a "memory" the model can look back at to catch its own mistakes.

---

Lesson Exercise

The Logic Trap:

1. Ask an LLM: "Sally has 3 brothers. Each of her brothers has 2 sisters. How many sisters does Sally have?" (The answer is 1—herself! But AI often says 2 or 6).
2. If it fails, analyze the response. Did it start calculating too fast without "thinking"?
3. Now, tell it: "Think step-by-step about who the 'sisters' are in relation to Sally."

Observation: By slowing the model down, you are forcing it to bridge the Reasoning Gap.

---

Summary

In this lesson, we established:

• LLMs excel at System 1 (Intuitive/Pattern) tasks.
• LLMs struggle with System 2 (Logical/Deliberate) tasks.
• The lack of a true World Model means AI relies on correlation, not deep understanding.

Next Lesson: We dive deeper into the difference between correlation and reality. We'll learn about Correlation vs. Causation and why LLMs struggle to understand "Why" things happen.