Redundant Context Injection: The RAG Token Drain

In the world of Retrieval-Augmented Generation (RAG), the philosophy is simple: "Give the model the documents it needs to answer the question." However, many developers interpret this as "Give the model AS MANY documents as will fit in the context window."

This leads to the third major source of token waste: Redundant Context Injection.

If your vector database returns five chunks that all essentially say the same thing, sending all five to the model is like paying five different people to tell you the same news. It costs more, takes longer, and doesn't make you any smarter.

---

1. The Anatomy of Redundancy

Why does redundancy happen?

1. Overlap in Chunking: When you split documents, you often use "Overlapping Windows" (e.g., 500 characters with 100 characters of overlap). If a search returns two adjacent chunks, you are sending 200 tokens of duplicate text.
2. Duplicate Source Material: Many corporate wikis or Slack channels have the same info repeated in five different places.
3. Semantic Similarity without Uniqueness: "How do I reset my password?" might return five different help articles that all have the same three steps.

graph TD
    Q[User Query] --> V[(Vector DB)]
    V --> C1[Chunk 1: 'Click Settings']
    V --> C2[Chunk 2: 'Click Settings... then Profile']
    V --> C3[Chunk 3: 'Settings is in the top right']
    
    subgraph "Token Waste"
        C1
        C2
        C3
    end
    
    C1 & C2 & C3 --> LLM[Large Language Model]

---

2. The Multi-Model Mirror Effect

If you are using an "LLM-as-a-Judge" or a multi-agent system, the problem becomes exponential.

• Agent 1 reads 5 chunks (2,000 tokens).
• Agent 1 summarizes them into 1,000 tokens.
• Agent 2 reads the original 5 chunks AND the summary (3,000 tokens).

By the time you reach the final output, you have processed the same "Settings" information three or four times.

---

3. Solving Redundancy: The "Deduplication Wrapper"

In your FastAPI backend, you should never pass raw results from a vector database directly to the model. You should implement a Semantic Deduplicator.

Python Code: Context Deduplication

from sentence_transformers import util

def deduplicate_chunks(query, chunks, threshold=0.85):
    """
    Remove chunks that are semantically too similar to each other
    to avoid wasting tokens.
    """
    unique_chunks = []
    
    for i, current_chunk in enumerate(chunks):
        is_duplicate = False
        for saved_chunk in unique_chunks:
            # Measure similarity between the new chunk and existing ones
            # Using a simple string overlap or a fast embedding model
            similarity = calculate_similarity(current_chunk, saved_chunk)
            
            if similarity > threshold:
                is_duplicate = True
                print(f"Skipping redundant chunk {i}")
                break
        
        if not is_duplicate:
            unique_chunks.append(current_chunk)
            
    return unique_chunks

def calculate_similarity(a, b):
    # For token efficiency, even a simple set() comparison 
    # of words can detect 80% of redundant chunks.
    set_a = set(a.lower().split())
    set_b = set(b.lower().split())
    overlap = len(set_a.intersection(set_b)) / min(len(set_a), len(set_b))
    return overlap

---

4. The "Max-K" Trap

Many RAG tutorials suggest top_k=10. Why? Because 10 sounds better than 3.

But in production:

• 10 chunks ≈ 4,000 tokens.
• 3 chunks ≈ 1,200 tokens.

If 3 chunks contain the answer, the other 7 are pure waste.

The Solution: Use a Re-ranker (like Cohere or BGE).

1. Retrieve 20 chunks (Recall).
2. Use a cheap re-ranker to find the Top 3 most distinct and relevant chunks (Precision).
3. Send only those 3 to the expensive LLM.

---

5. Token Savings with "Long-Context" Models

Some developers think: "Claude 3 has a 200k window, so I'll just send everything. It's easier."

The Financial Trap:

• 1M Input Tokens on Claude 3.5 Sonnet = $3.00.
• 10 Queries with 100k context = 1M tokens.
• Cost: $3.00 for 10 users.

If your app gets 10,000 users, you just spent $3,000 on one day of data processing that could have been handled by a $50 RAG system.

---

6. Architecture: Context-Aware Summarization (LangGraph)

Instead of passing raw text, use a "Compressor Node" in your agentic graph.

graph LR
    A[Doc Retrieval] --> B[Deduplication Node]
    B --> C[Summarization Node]
    C --> D[Final Reasoning Node]
    
    subgraph "Token Counts"
        A_T[5,000 tokens]
        D_T[500 tokens]
    end

By adding 100ms of "Summary" time, you save 90% of the input tokens for every downstream node in your graph.

---

7. Summary and Key Takeaways

1. Similarity != Utility: Just because a document is similar to the query doesn't mean it's useful to the model if you already have that info.
2. Deduplicate early: Use string sets or light embeddings to remove redundant chunks before the "Final" LLM call.
3. Re-ranking is cheaper than Token Waste: Spending $0.01 on a re-ranker to save $0.50 on tokens is an 50x ROI.
4. Beware of Large Windows: Just because you can fit a document doesn't mean you should pay for it.

In the next lesson, Large, Unfiltered Documents, we explore the cost of "Laziness" in data ingestion and how to prune your sources for efficiency.

---

Exercise: The Redundancy Audit

1. Retrieve the top 5 chunks for a common query in your app.
2. Manually read them. How many of them provide new information that wasn't in Chunk 1 or 2?
3. If the answer is "none," try reducing your top_k to 3.

• Measure the Accuracy (does the answer change?) vs the Cost (how many tokens were saved?).
• Usually, you will find that accuracy stays the same while cost drops by 60%.

---

Congratulations on completing Module 2 Lesson 3! You are now a RAG optimizer.