RAGembeddingsvectorscosine similaritysemantic searchmodel selection

Embeddings

An embedding model maps text to a vector so that semantically similar texts land close together; retrieval becomes nearest-neighbour search.

Interview question
Progress

Text as a point in space

An embedding model is a neural network that reads a piece of text and outputs a fixed-length list of numbers — typically 384 to 3072 floats. The model is trained so that texts with similar meaning produce vectors that point in similar directions. "How do I reset my password?" and "forgot login credentials" end up near each other even though they share no words.

That single property turns semantic search into geometry. Embed every chunk once at ingestion, embed the query at request time, and find the chunks whose vectors are closest to the query vector. No keyword matching, no synonyms lists, no stemming rules.

What the vector encodes is whatever the training objective rewarded. General-purpose models are trained on web-scale pairs of related texts (question/answer, title/body, paraphrases) and are good at *topical* similarity. They are not trained to know that error code E4021 differs from E4012, or that your internal product names mean anything.

Cosine similarity, with numbers

The standard similarity measure is cosine similarity: the cosine of the angle between two vectors, computed as the dot product divided by the product of the lengths. It ranges from −1 to 1; for embedding models it is usually between 0 and 1, and most models normalise vectors to unit length so cosine equals the plain dot product.

A tiny example in three dimensions. Query q = [0.9, 0.1, 0.0]. Chunk A [0.8, 0.2, 0.1], chunk B [0.1, 0.2, 0.9]. Dot products: q·A = 0.72 + 0.02 + 0 = 0.74; q·B = 0.09 + 0.02 + 0 = 0.11. After dividing by lengths (≈0.906 × 0.831 for A, ≈0.906 × 0.927 for B), cosine(q, A) ≈ 0.98 and cosine(q, B) ≈ 0.13. A is retrieved. Real vectors have hundreds of dimensions, but the arithmetic is identical.

Absolute scores are not comparable across models: one model's "0.8" is another's "0.5". Never hard-code a similarity threshold you copied from a blog post; measure the score distribution on your own relevant and irrelevant pairs and pick the cutoff from that.

Cosine similarity and a brute-force top-k — correct, and fine up to ~100k vectors.
1function cosine(a: number[], b: number[]): number {
2  let dot = 0, na = 0, nb = 0;
3  for (let i = 0; i < a.length; i++) {
4    dot += a[i] * b[i]; na += a[i] * a[i]; nb += b[i] * b[i];
5  }
6  return dot / (Math.sqrt(na) * Math.sqrt(nb));
7}
8 
9function topK(query: number[], chunks: { id: string; vec: number[] }[], k = 5) {
10  return chunks
11    .map((c) => ({ id: c.id, score: cosine(query, c.vec) }))
12    .sort((x, y) => y.score - x.score)
13    .slice(0, k);
14}

Dimensions and model selection

More dimensions means more capacity to separate meanings, but also more storage, slower distance computation, and larger index memory. 1M chunks at 1536 float32 dimensions is about 6 GB of raw vectors; at 384 dimensions it is 1.5 GB. Some models support truncating vectors to fewer dimensions with modest quality loss (Matryoshka-style training); quantising to int8 or binary cuts memory further at some recall cost.

Choose a model by: retrieval quality *on your data* (build a 50-pair golden set and measure recall@k), max input length (a 512-token model silently truncates a 900-token chunk), language coverage, latency and cost per token, and whether you can run it locally. Public leaderboards rank models on generic benchmarks; a model that wins there can lose on your legal contracts or your codebase.

Whatever you pick, the same model must embed both the chunks and the queries. Vectors from different models live in unrelated spaces, and even a version bump can move the space. Store the model name and version with the index, and treat a model change as a full re-embed.

  • Measure on your corpus; benchmark rank is a prior, not a decision.
  • Check max tokens: truncated chunks embed only their first half.
  • Pin the model version; re-embed everything on change.
  • Smaller dimensions are fine when a reranker cleans up the top candidates.

Domain mismatch

General embedding models know the web. They do not know that in your company "Atlas" is a billing service, that RC-17 is a rate-card revision, or that two drug names differ by one letter and treat entirely different conditions. On such corpora, dense retrieval alone confuses lookalikes and misses exact identifiers — which is the main argument for adding a sparse index (Dense, Sparse & Hybrid Retrieval).

Mitigations in ascending cost: enrich chunks with expanded names and synonyms before embedding; add a keyword index for the identifiers; fine-tune an embedding model on pairs mined from your own logs (query → clicked passage); or use a domain-specific model (code, biomedical, legal) if one exists. Fine-tuning is rarely the first move — most teams get more from fixing chunking and adding hybrid search.

Embedding the question vs the passage

A question and its answer are not paraphrases. "What is the refund window for annual plans?" and "Refunds on annual plans must be requested within 14 days." are semantically linked but stylistically different — one is interrogative and short, the other declarative and specific. Many models are trained as asymmetric retrievers: they expect a query-style input on one side and a passage-style input on the other and use different instruction prefixes (such as query: / passage:) for each. Omitting the prefix degrades recall noticeably.

Two practical techniques close the gap further. HyDE (hypothetical document embedding) asks the LLM to write a plausible answer paragraph, embeds that, and searches with it — the fake answer looks more like a real passage than the question did. Question-side indexing does the opposite: generate a few likely questions per chunk at ingestion, embed those, and map them back to the chunk, so queries are matched against queries.

Both cost extra LLM calls (HyDE at query time, question generation at ingestion). Use them when a retrieval eval shows the query/passage gap is the problem, not by default.

Key points

  • An embedding is a fixed-length vector; similar meaning → nearby vectors.
  • Cosine similarity = dot product of unit vectors; scores are only comparable within one model.
  • Chunks and queries must be embedded by the same model version.
  • Pick the model with a retrieval eval on your own data, not from a leaderboard.
  • General models miss exact identifiers and domain lookalikes — pair dense with sparse.
  • Use asymmetric prefixes; consider HyDE or question-side indexing when queries and passages differ in form.

When to use — and when not to

Use it when
  • Natural-language queries where wording differs from the document.
  • Multilingual corpora where users query in a different language from the docs.
  • Semantic deduplication and clustering of documents.
  • Any retrieval where synonyms and paraphrase matter more than exact tokens.
Avoid it when
  • Exact-match lookups (IDs, SKUs, error codes) — use a keyword index or a database.
  • Tiny corpora where a keyword grep is faster and fully predictable.
  • When you cannot re-embed on model change — vectors are not portable.

Failure modes

  • Query embedded with one model, chunks with another; recall collapses silently.
  • Chunks longer than the model's max tokens are truncated; the second half is never findable.
  • Threshold copied from documentation drops relevant results on a different model.
  • Lookalike identifiers (E4021 vs E4012) retrieved interchangeably.
  • Asymmetric model used without query/passage prefixes.
← Ingestion: Parsing & ChunkingVector Storage →