Planning in Agentic Systems: CoT, ReAct, and Tree-of-Thought

A common frustration when working with LLMs is the "Rush to Response." By default, a model wants to provide an answer as quickly as possible. However, for an agent, an immediate answer is often a hallucinated or incomplete answer. To build a robust Gemini ADK agent, we must teach the model how to Plan before it acts.

In this lesson, we will explore the three most powerful planning paradigms in modern AI: Chain-of-Thought (CoT), ReAct, and Tree-of-Thought. We will learn why "Thinking Step-by-Step" is a technical requirement, not just a suggestion, and how to implement these patterns in your agents.

1. Why Models Need to Plan

Large Language Models do not "think" in the way humans do. They predict the next most likely token based on mathematical probabilities.

The Problem: Immediate Inference

If you ask a model to "Build a travel itinerary for Italy," it might immediately start listing cities. Midway through, it realized it forgot to check the flight prices or the customer's budget.

The Solution: Latent Reasoning

By forcing the model to generate a "Plan" first, we are providing it with a Working Memory (the tokens of the plan itself) that it can "read" during the actual execution. This transforms a one-shot guess into a multi-step logical process.

2. Chain-of-Thought (CoT): The "Think out Loud" Pattern

CoT is the simplest but most effective planning technique. It encourages the model to generate intermediate reasoning steps instead of just the final answer.

Zero-Shot CoT: Adding the phrase "Think step-by-step" to your prompt.
Manual CoT: Defining a specific block (e.g., <thought> ... </thought>) where the agent must justify its decisions.

Why it works: Each "thinking" token provides a new bit of context that influences the probability of the next token. The more rigorous the thinking, the more accurate the result.

3. The ReAct Pattern (Reason + Act)

ReAct is the "Gold Standard" for agentic behavior. It combines the reasoning of CoT with the action-taking of tools.

The ReAct Cycle:

Thought: "I need to find the population of France. I will use the Search tool."
Action: call search(q='population of France')
Observation: "67 million people."
Repeat: "Now I need to calculate 10% of that..."

graph TD
    A[User Goal] --> B[Reasoning: What to do?]
    B --> C[Action: Call Tool]
    C --> D[Observation: Result]
    D --> E{Goal Met?}
    E -->|No| B
    E -->|Yes| F[Final Answer]
    
    style B fill:#4285F4,color:#fff
    style C fill:#F4B400,color:#fff

In the Gemini ADK, the enable_automatic_function_calling=True setting implements a specialized version of the ReAct pattern for you.

4. Tree-of-Thought (Exploring Multiple Paths)

For extremely complex and high-stakes tasks (e.g., medical diagnosis or architectural design), a linear plan isn't enough. Tree-of-Thought allows the agent (or a group of agents) to explore multiple different paths simultaneously and "prune" the ones that don't look promising.

The Workflow:

Enumerate: Generate 3 different plans.
Evaluate: Assess the likelihood of success for each plan.
Execute: Follow the most promising path.

5. Self-Reflection and Correction

A critical part of planning is the ability to realize when you are wrong.

Self-Correction: If a tool returns an error (e.g., "File not found"), the agent shouldn't just crash. It should generate a new "Thought": "The file was not found in the current directory. I will search the sub-directories instead."
Self-Critique: After a plan is finished, the agent can be instructed to review its own work: "Is there any part of this plan that violates our safety constraints?"

6. Implementation: A Manual ReAct Loop in Python

While the Gemini SDK handles much of this, writing it manually once is deeply educational.

import google.generativeai as genai

model = genai.GenerativeModel('gemini-1.5-flash')

def re_act_agent(goal: str):
    # The 'Mental State' of the agent
    history = [
        {"role": "user", "parts": [
            f"GOAL: {goal}\n"
            "INSTRUCTION: Use a 'THOUGHT:' block and an 'ACTION:' block. "
            "Wait for my 'OBSERVATION:' before proceeding."
        ]}
    ]
    
    for _ in range(3): # Limit to 3 reasoning turns
        # 1. GENERATE THOUGHT & ACTION
        response = model.generate_content(history)
        print(f"\nAGENT SAYS:\n{response.text}")
        
        # 2. Add response to history
        history.append({"role": "model", "parts": [response.text]})
        
        # 3. MOCK EXECUTION (Simulator)
        if "SEARCH" in response.text:
            observation = "Result: The weather in London is 15 degrees."
            print(f"SYSTEM: {observation}")
            history.append({"role": "user", "parts": [f"OBSERVATION: {observation}"]})
        else:
            # If no tool requested, assume goal is reached
            break
            
    return "Agent task complete."

# re_act_agent("Find the weather in London and suggest a coat.")

7. Directing the Reasoner: System Prompts for Planning

The best way to influence the planning process is through the System Prompt.

Effective Planning Constraints:

"Always prioritize safety over speed."
"If a task can be parallelized, do so."
"Show your planning steps in a hidden thought block."

8. Summary and Exercises

Planning is the Framework of Intelligence.

CoT provides the working memory.
ReAct provides the loop of logic and action.
Self-Reflection provides the resilience to error.

Exercises

Planning Breakdown: You want an agent to "Review 10 resumes and pick the best one." Write out the "Thought-Action-Observation" steps the agent would take.
Prompt Refinement: Take a query like "How do I fix a leaky faucet?" and write it using Chain-of-Thought instructions (e.g., "Think step-by-step from the perspective of a plumber"). Notice if the explanation changes.
Error Handling: If an agent's plan is "1. Search for X, 2. Open X, 3. Read X," but Step 1 reveals that "X" is actually a group of 5 files (Y1, Y2, Y3, Y4, Y5), how should the agent's Execution Plan change in Turn 2?

In the next lesson, we will look at Execution Plans, moving from the "Theory of Thought" to the "Mechanics of Multi-Step Tasks."