Rolling Updates: Zero-Downtime Re-indexing

How do you update 10 million vectors without the user noticing? If you simply stop the database, clear it, and start re-uploading, your app will be broken for hours (or days).

In this lesson, we learn the Rolling Update and Dual-Write patterns for seamless vector data management.

---

1. The "Background Rebuild" Pattern

This is the most common rolling update strategy:

1. Copy: Create a new index (as discussed in Module 17.3).
2. Backfill: Run a script to read all historical documents and push them to the new index.
3. Delta Ingestion: While the backfill is running, new documents are still being added to the old index. You must track these "New" items and push them to the new index as well.
4. Cutover: Once the backfill catches up to the present, you switch query traffic to the new index.

---

2. The "Dual-Write" Pattern

If you are move to a new architecture entirely, you implement Dual-Writing.

• The Logic: Whenever a document is created or updated, your backend sends it to two locations:
  1. The "Old" (Active) Vector DB.
  2. The "New" (Scaling) Vector DB.
• Goal: Eventually, the "New" DB will have all the fresh data. You then backfill the historical data, and you're ready to switch.

---

3. Implementation: The Safe Upsert (Python)

def update_document(doc_id, text):
    # 1. Update the Main Index (Current Prod)
    prod_index.upsert(id=doc_id, vector=old_model.encode(text))
    
    # 2. ALSO update the Migration Index (Future Prod)
    migration_index.upsert(id=doc_id, vector=new_model.encode(text))
    
    # This prevents the two indexes from drifting apart during the move!

---

4. The "Partial Update" challenge

Managed databases like Pinecone support "Metadata-only" updates.

• If you only want to change a metadata field (e.g., is_public: true -> false), you don't need to re-index the vector.
• Using index.update(id=..., set_metadata=...) is 100x faster than deleting and re-upserting the vector.

---

5. Summary and Key Takeaways

1. Avoid the Clear-All: Never delete_all in production.
2. Dual-Writing: Write to both old and new indexes during a migration period to maintain parity.
3. Atomic Swaps: Use aliases (Module 17.3) for the final cutover to ensure zero downtime.
4. Metadata Over Vectors: If you only need to change a tag, use a metadata-only update to save compute and time.

In the next lesson, we’ll look at the nightmare scenario: Incident Response.

---

Congratulations on completing Module 17 Lesson 4! You are now move data like a senior engineer.