Memory Management Strategies

An agent that "Remembers Everything" eventually becomes slow, expensive, and confused. This is the "Memory Bloat" problem. If every interaction adds 1,000 tokens to the long-term store, the agent will eventually be buried under a mountain of irrelevant data.

In this lesson, we will learn how to manage the lifecycle of a memory—from creation to Consolidation and, eventually, Deletion.

---

1. The Strategy: Recency vs. Frequency vs. Relevance

When an agent retrieves a memory, it must balance three factors:

1. Recency: "This event happened 5 minutes ago." (High importance).
2. Frequency: "The user has mentioned this 50 times in 10 days." (High importance).
3. Relevance: "This event is semantically similar to the current question." (High importance).

The Weighting Formula: Final_Score = (Relevance * 0.5) + (Recency_Score * 0.3) + (Frequency_Score * 0.2)

---

2. Memory Consolidation (The "Sleep" Cycle)

Just like humans, agents should "Consolidate" their memories during idle time.

The Background Worker

At 2 AM, your server runs a "Consolidator Agent":

1. It reads 100 small memories from the day.
2. It groups them: "The user talked about their dog 5 times."
3. it replaces the 100 small entries with one high-level summary: "User has a dog. They take it to the park on Tuesdays."
4. Benefit: You have saved 99% of your "Vector Space" while keeping the "Semantic Fact."

---

3. Tiered Storage (The "Warmth" of Memory)

• Hot Memory (RAM/Redis): Extremely small. Stores the current user intent and the last 3 messages.
• Warm Memory (Postgres): Stores the summary of the last 10 sessions.
• Cold Memory (S3/Pinecone): Stores every raw message ever sent (for audit purposes, not for active reasoning).

---

4. Relevance Filtering (Top-K vs. Threshold)

When you query a vector DB, don't just take the "Top 5."

• Top-K: Always gives you 5 items, even if they are total garbage.
• Threshold: Only takes items with a similarity score > 0.8.
• Recommendation: Always use a Threshold. It's better for the agent to say "I don't remember" than to retrieve a "Near-miss" that leads to a hallucination.

---

5. Metadata-Driven Expiry (TTL)

Not all memories are permanent.

• Project-Specific Memory: "The password for the dev server is XYZ." -> Delete when Project Status is 'Done'.
• User Preference: "I prefer dark mode." -> Keep Forever.
• Transient Fact: "I am currently at the airport." -> Delete in 24 hours.

How? Add a ttl (Time to Live) field to your memory schema.

---

6. Implementation: The "Reference Counter"

If an agent retrieves a memory and the user says "That's wrong," the agent should call a decay_memory tool.

• This reduces the Confidence Score of that specific vector.
• If the score hits 0, the memory is deleted.
• This allows a "Self-Correcting" memory system.

---

Summary and Mental Model

Think of Memory Management like Cleaning your House.

• If you never throw anything away, you can't find your keys (The Retrieval Problem).
• If you throw everything away, you lose your history (The Context Problem).

A healthy agent has a "Clean Desk" policy: Keep what's useful, archive what's old, and throw away the trash.

---

Exercise: Storage Strategy

1. The Choice: A user tells the agent: "I'm looking for a gift for my wife's birthday next Friday."
   • Should this be stored with a TTL?
   • If so, what should the expiry date be?
2. Consolidation: Write a prompt for a "Consolidator Agent" that takes 5 raw chat messages and converts them into a single Entity Update.
3. Optimizing: Why is "Recency" often more important for a Coding Agent than a History Agent? Ready for the most important legal part? Next lesson: Privacy and the "Right to Forget."