Avoiding Recursive System Prompts: The State Trap

In conversational AI, the system message is meant to be the "Foundation." But as a conversation continues, or as multiple agents collaborate, we often find ourselves repeating that foundation over and over. This is Recursive Prompting, and it is a massive waste of tokens.

In this lesson, we learn how to implement Instruction Inheritance, how to use "Compressed State," and how to manage global rules without duplicating them in every message.

---

1. The Recursive Problem

Imagine a 20-turn conversation. If your system prompt is 500 tokens, by Turn 20, you have sent 10,000 tokens of the exact same instructions.

Even worse, in a multi-agent system (LangGraph), Agent A might pass its entire context to Agent B, including Agent A's original system prompt. Agent B then appends its own system prompt. This results in "Instruction Accumulation."

graph LR
    A[Agent A] -->|Full History| B[Agent B]
    B -->|Bigger History| C[Agent C]
    
    subgraph "Token Bloat"
        B_P[System A + History + System B]
        C_P[System A + History + System B + History + System C]
    end

---

2. Solving with "Instruction Isolation"

Instead of sending the full system prompt every time, you should categorize your instructions into:

1. Immutable Goals: Send every time (e.g., "Safety Rules").
2. Current Context: The last 3-5 messages.
3. Task Delta: The specific instruction for the current turn.

The "Summary State" Pattern

Every 5 turns, have a cheap model (Module 3.2) summarize the conversation history. Remove the old verbatim messages and the redundant system prompts from the history. Replace them with:

"State: [Summary of previous facts and current task status]."

---

3. Multi-Agent Inheritance (LangGraph)

In LangGraph, you have a shared State. Do not put instructions in the state. Put Data in the state.

The Correct Way: Each node (agent) has its own local SystemMessage. It reads the State to get the latest variables but doesn't "re-read" the instructions of the previous agents.

Python Code: Clean State Passing

# Bad Pattern
def node_a(state):
    prompt = f"System: {GLOBAL_SYSTEM}\nUser: {state['msg']}"
    # Aggregates EVERYTHING into the next message
    return {"history": state['history'] + [prompt]}

# Good Pattern
def node_a(state):
    # Only pass DATA, not instructions
    facts = state['facts']
    return {"facts": facts + ["New fact from AI"]}

---

4. Using "User" as the Context Holder

For long conversations, move some of the instructions into the "User" block as metadata. Modern models (Claude 3.5) are efficient at distinguishing between "Instructions from the user" and "System instructions."

Pattern: System: You are a helpful assistant. User: [META: Output must be JSON] Can you help me find...

This allows you to change specific rules per-message without resetting the entire system "Brain."

---

5. Summary and Key Takeaways

1. Prune History: Don't sendTurn 1's system prompt in Turn 20.
2. State != Prompt: Keep your architectural logic in your Python code, and only send the Result to the model.
3. Summarize Regularly: Compress the "History" every few turns to keep the "Input Token" count low.
4. Agentic Hygiene: Ensure agents pass data objects, not raw prompt strings, to their peers.

In the next lesson, Token-Efficient Formatting, we explore why Markdown and XML are the secret weapons of the efficiency-first developer.

---

Exercise: The History Pruner

1. Write a Python function that takes a list of 20 chat messages.
2. If the total token count is > 4,000, the function should:
   • Keep the first message (The System Prompt).
   • Keep the last 3 messages (The Context).
   • "Zip" the middle 16 messages into a single summary string.
3. Observe the Token Savings.

• Usually, this "Sliding Window with Summary" technique saves 50-80% of tokens in long-running sessions.

---

Congratulations on completing Module 4 Lesson 3! You are now a master of conversational flow.