Retrieval-augmented generation (RAG) lets a language model answer questions using your documents instead of guessing. Done well, it gives grounded answers with citations. Done poorly, it confidently makes things up.
Retrieval quality is everything
Most "the AI is wrong" bugs are retrieval bugs. Invest in:
- Chunking that respects document structure — headings, tables and lists.
- Hybrid search combining keywords with embeddings.
- Permission filtering so users only retrieve what they are allowed to see.
Always show sources
Every answer should cite the passages it used. Citations build trust and make errors easy to spot and fix.
Build an evaluation set early
Collect 50–100 real questions with expected answers. Run them on every change to prompts, models or chunking. Without an eval set you are tuning by feel.
for case in eval_set:
answer = assistant.ask(case.question)
score = grade(answer, case.expected, case.sources)
results.append(score)
print(f"accuracy: {mean(results):.1%}")Guardrails
Treat retrieved text as untrusted input, keep a human in the loop for actions with real consequences, and log enough to debug without storing more personal data than necessary.
