Get Started

RAG Evaluation: Precision, Recall, MRR, nDCG, Faithfulness, Answer Relevance

If you cannot grade your retrieval and your answer at the same time, you cannot ship the system. The two are not the same metric and the same prompt fails on different axes.

90 minutesBuildPythonMembers

Learning Objectives

  • Compute four retrieval metrics from gold qrels: precision@k, recall@k, MRR (mean reciprocal rank), and nDCG@k.
  • Compute two answer-grade metrics: faithfulness (every claim grounded in retrieved context) and answer relevance (the answer addresses the question).
  • Build a fixture qrels file (queries, gold doc ids, gold answer text) that the eval reads end to end.
  • Read the metric values to diagnose where a pipeline is failing: retrieval, ranking, generation, or grounding.