Managing Agentic State at Scale: Distributed Sessions and Redis

In a prototype, state is easy. You have a chat object in memory, and as long as the script is running, the agent remembers everything. In production, this "In-Memory" state is a liability. Your cloud provider might kill your server instance at any time, or a Load Balancer might send the user's second message to a different server than the first one.

To scale Gemini ADK apps, we must follow the Stateless Worker Pattern. The server should not "remember" the user; instead, the server should "fetch" the user's memory from a database on every single request. In this lesson, we will explore distributed state management using Redis and the patterns for "Hydrating" your agents at scale.

---

1. The Challenge of "Memory" on Multiple Servers

Imagine 2 servers (A and B) and 1 User.

• Turn 1: User says "Hi, I'm Sudeep" to Server A. Server A stores "Sudeep" in its RAM.
• Turn 2: User says "What is my name?" to Server B.
• The Fail: Server B has no idea who Sudeep is.

The Solution: Externalized State

Every turn, the agent follows this lifecycle:

1. Fetch: Retrieve chat history from Redis using the session_id.
2. Initialize: Create the Gemini chat object and feed it the history.
3. Process: Execute the turn.
4. Save: Write the new history (including the latest turn) back to Redis.

---

2. Using Redis for High-Speed State

Redis is the industry standard for this task because it is an in-memory database with microsecond latency.

• Data Structure: Use the LIST type in Redis to store the serialized JSON of each turn.
• TTL (Time to Live): Automatically delete sessions after 48 hours to save costs and respect data privacy.

---

3. History Hydration Pattern

When you fetch history from a database, it's just raw text or JSON. You must "Hydrate" it into the format that the GenerativeModel.start_chat method expects.

The Gemini Format:

Each turn must be a part of a list with role ("user" or "model") and parts.

# Hydration Logic
def hydrate_history(raw_json_list):
    return [
        {"role": item['role'], "parts": [item['text']]}
        for item in raw_json_list
    ]

---

4. Context Pruning (Memory Management)

As a conversation grows to 100 turns, your history might reach 500k tokens. Sending 500k tokens to Gemini on every "Turn 101" is:

1. Slow (Latency).
2. Expensive (Cost).

The Pruning Strategy:

Before saving back to Redis, check the length of the history. If it exceeds your limit (e.g., the last 20 turns), Summarize the oldest 10 turns into a single "Memory Block" and delete the raw logs for those turns. This allows the agent to maintain "Knowledge" without "Baggage."

graph TD
    A[New Turn: History = 20 turns] --> B{Too Big?}
    B -->|Yes| C[Agent Summarizes turns 1-10]
    C --> D[Delete turns 1-10]
    D --> E[Insert Summary at Turn 1]
    E --> F[Session is now 'Lean']
    B -->|No| G[Save Raw History]
    
    style C fill:#4285F4,color:#fff

---

5. Implementation: Redis-Backed Agent

import redis
import json
import google.generativeai as genai

# Setup
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
model = genai.GenerativeModel('gemini-1.5-flash')

def get_response(session_id, user_message):
    # 1. FETCH
    history_json = r.get(f"history:{session_id}")
    history = json.loads(history_json) if history_json else []
    
    # 2. HYDRATE & INITIALIZE
    chat = model.start_chat(history=history)
    
    # 3. PROCESS
    response = chat.send_message(user_message)
    
    # 4. PERSIST
    # chat.history contains the UPDATED list
    # We serialize the protobuf objects to simple JSON/dicts
    serialized_history = [
        {"role": m.role, "parts": [p.text for p in m.parts]}
        for m in chat.history
    ]
    r.set(f"history:{session_id}", json.dumps(serialized_history))
    
    return response.text

---

6. Snapshotting: Saving "Points in Time"

For long-running autonomous research tasks (which might take 10 minutes), we use Snapshots. Instead of saving every turn, we save the entire "Agent State" (including its current sub-goals and findings) to a database so that if the server crashes, the agent can resume from exactly where it left off.

---

7. Multi-Region State and Latency

If you have users in London and Tokyo, you need Redis in both regions. Use Redis Global Data Stores to replicate the session_id so a user can travel between regions and stay logged in to their identical agent state.

---

8. Summary and Exercises

Scaling an agent requires Decoupling State from Compute.

• Stateless Workers allow for infinite horizontal scaling.
• Redis provides the ultra-low latency needed for "Hydrating" history.
• Pruning and Summarization keep the context window efficient and cheap.
• Serialized History is the bridge between the database and the Gemini SDK.

Exercises

1. Hydration Logic: Write a Python function that converts a list of chat.history messages into a list of Python dictionaries that can be saved to a JSON file.
2. State Recovery: Imagine your Redis database fails. What is the experience for a user who was in the middle of a 10-turn planning session? How could a "Fallback to SQL" strategy help?
3. Efficiency Math: If one turn is 500 tokens, how many turns can a user take before they hit a $5.00 cost limit? How does Context Pruning extend this limit?

In the next lesson, we will look at Security and Governance, exploring how to protect these massive stores of user data.