RAG Engineering
Ingestion, embeddings, storage, retrieval, reranking, grounding, evaluation.
Retrieval-Augmented Generation fetches relevant passages at query time and puts them in the prompt so the model answers from evidence instead of from memory.
Turning raw files into clean, well-bounded, well-labelled chunks is where most RAG quality is won or lost.
An embedding model maps text to a vector so that semantically similar texts land close together; retrieval becomes nearest-neighbour search.
Where embeddings live: a plain database with a vector column, a dedicated vector store, or a search engine — and the ANN indexes that make nearest-neighbour search fast.
Dense vectors capture meaning; BM25 captures exact terms; hybrid retrieval fuses both so neither paraphrases nor identifiers are missed.
Restricting retrieval by tenant, permission, recency, or type is done with metadata filters — and where the filter runs decides both correctness and recall.
A reranker re-scores a small candidate set with a more expensive model so the few chunks that reach the LLM are the right ones — it fixes precision, not recall.
Turning ranked chunks into a prompt: order, deduplicate, fit the token budget, and instruct the model to answer only from what it was given — or to say it cannot.
A citation is a verifiable pointer from a claim to a retrieved span; the system, not the model, must check that it points at real text.
Measure retrieval and generation separately: recall@k, precision, MRR for the retriever; faithfulness and answer correctness for the generator — on a golden set you built from real queries.