Embeddings

What an embedding is

For text-embedding-3-small, an embedding is a list of 1536 floating-point numbers between roughly -1 and 1. Same text → same vector, always. Different texts → different vectors. Texts with similar meaning → similar vectors.

"what is ownership in rust"      → [0.012, -0.034, 0.567, ..., -0.089]
"explain rust ownership"         → [0.011, -0.031, 0.572, ..., -0.085]   (very similar)
"how do I cook pasta"            → [-0.42, 0.81, -0.15, ..., 0.33]       (very different)

The embedding model was trained on billions of text pairs so that semantically similar text produces vectors close to each other in this 1536-dim space. You don’t see or interact with the training; you just call the model and get back numbers.

The intuition: points in space

Picture each embedding as a point in a 1536-dimensional space. (You can’t visualize 1536 dimensions, but the math works the same as 2D.) In 2D the picture would be:

       |
       |   "rust ownership" •  • "rust borrowing"
       |          • "rust lifetimes"
       |
       |
       |
       |                                  • "javascript closures"
       |                              • "javascript hoisting"
       |________________________________________________
                                                  • "kitchen recipes"

Topics cluster. Rust stuff groups together. JavaScript stuff groups together. Pasta is off in its own corner. Cosine similarity is the angle between two vectors - cosine of 1.0 = identical direction, 0.0 = unrelated, -1.0 = opposite.

Vector similarity search = “find the points closest to my query point.” Embed the question, look in the space for the nearest stored chunks, return them.

Why this beats keyword search for RAG

The trade-off: embedding search is slower than keyword search (per-vector dot product vs inverted index) and you need to keep the vector store in sync with the source files.

For small corpora (under a million chunks), embedding search wins on quality. For Wikipedia-scale, hybrid approaches (BM25 keyword + embedding rerank) win.

Where my RAG app embeds text

Two distinct places:

Point 1: ingest time (every chunk)

underlying = OpenAIEmbeddings(model="text-embedding-3-small", api_key=...)
store = RedisStore(redis_url=..., namespace="rag-app")
embeddings = CacheBackedEmbeddings.from_bytes_store(...)
Chroma.from_documents(documents=chunks, embedding=embeddings, ...)

Each chunk → vector → stored in Chroma alongside chunk text + metadata. After ingest, Chroma has N triples of (text, vector, metadata).

This is the expensive offline step. My app’s 235 chunks = ~$0.002 + ~15 seconds. Redis caches it so re-ingest is near-free.

Point 2: query time (the user’s question)

docs = await _vectorstore().asimilarity_search(question, k=k)

Under the hood:

1. Embed the question (one OpenAI call)
2. Compute cosine similarity between the question vector and every stored vector
3. Return the top-k chunks (closest in vector space)

Cost per query: ~$0.00002 for the embedding, microseconds for the vector search.

The “same embedder” non-negotiable

If you embed documents with text-embedding-3-small and questions with text-embedding-3-large, the vectors live in incompatible spaces:

• Different dimensions (1536 vs 3072)
• Different mappings (even if dimensions matched, the two models would put the same text in different places)
• Cosine similarity between them is meaningless garbage

Even slightly different wrappers around the same model (different normalization, different tokenization) can produce subtly off vectors.

The fix is boring: use literally the same embedder instance everywhere. My RAG app does this by setting up OpenAIEmbeddings(model="text-embedding-3-small") identically in ingest.py and rag.py, both wrapped with CacheBackedEmbeddings the same way. Identical text → identical vectors → consistent vector space.

What text-embedding-3-small is specifically

OpenAI’s smaller, cheaper embedding model. Default choice for cost-conscious RAG in 2026.

For my app’s 235 chunks: under a cent and a few seconds total. Even a 10K-chunk corpus costs under a dollar.

What’s actually in chroma_db/chroma.sqlite3

After ingest, the SQLite file has (essentially) one table per collection with rows like:

When asimilarity_search runs, Chroma loads embeddings, computes cosine vs the question vector, sorts, returns the top-k Document objects (text + metadata, not the vector - you don’t need it after retrieval).

The vectors are the searchable index. The text + metadata are the payload you actually want.

What I noticed building my RAG app

• Keyword search would have caught maybe 60% of the same notes that embedding search returns. The 40% it’d miss are paraphrasings, synonyms, and “I asked about X using different words than my note used.”
• 1536 numbers per chunk × 235 chunks = ~1.5MB of vectors. Tiny. The whole chroma.sqlite3 file is under 4MB at this scale.
• The Redis embedding cache earns its keep most when I’m iterating on chunking (different chunk_size produces different chunks but most chunks would still hash to existing cached vectors if their text matches). At scale, the cache hit rate determines whether re-ingest is “instant” or “expensive.”
• I never have to look at the actual vectors. They’re an implementation detail; the abstraction is “embedder + similarity_search” and the math underneath stays invisible.