Why Vector Databases are Essential

Traditional databases (PostgreSQL, MySQL) were built for "Equal to" or "Greater than" operations on structured data. RAG requires "Similar to" operations on unstructured data. This is why Vector Databases (like Chroma) are essential.

Keyword Search vs. Semantic Search

Traditional Search (BM25/TF-IDF)

• Searches for exact word matches.
• Query: "The President of the United States"
• Won't find: "The Commander-in-Chief" or "POTUS" unless they are explicitly in the text.

Semantic Search (Vector)

• Searches for mathematical similarity in vector space.
• Query: "The President of the United States"
• Will find: "Commander-in-Chief" because they share a semantic neighborhood.

The Scalability Problem

Comparing one vector to 10 is fast. Comparing one vector to 10,000,000 is extremely slow (O(n) complexity). Vector databases solve this using Approximate Nearest Neighbor (ANN) algorithms like HNSW (Hierarchical Navigable Small Worlds).

Key Features of a Vector DB

1. Storage of Vectors & Content: They store the embedding alongside a "pointer" to the original text.
2. Metadata Filtering: They allow you to combine vector search with standard filters (e.g., WHERE year = '2024').
3. Point-in-time Updates: Easily add or delete documents without rebuilding the entire index.
4. Persistence: Unlike a simple array in memory, they handle disk storage and backups.

Why Chroma for this Course?

Chroma is building the "AI-Native" vector database. It is:

• Simple: Can run in-memory or on-disk with a few lines of code.
• Open Source: Full control over your data.
• Embedded: No need to manage a separate server during development.

Exercises

1. Compare the speed of a SQL LIKE %term% query vs. a vector search on a dataset of 10,000 rows.
2. Why is "Approximate" search acceptable in a RAG system? What is the trade-off?
3. What happens to a vector database if you double the number of dimensions in your embeddings?