Get Started

Classical Metrics

BLEU, ROUGE-L, F1, exact-match, accuracy. Five metrics that still account for most published LLM eval numbers. Implement each from first principles so you know what the number means.

90 minBuildPythonMembers

Learning objectives

  • Implement token-level exact-match, F1, and accuracy with explicit tokenisation rules.
  • Implement BLEU-4 from the ground up: modified n-gram precision, geometric mean over n equals 1 through 4, brevity penalty.
  • Implement ROUGE-L using longest common subsequence, with F-beta combination of precision and recall.
  • Dispatch on the metric_name field from lesson 70 so the runner stays metric-agnostic.
  • Pin the behaviour with reference vectors drawn from worked examples, not from a third-party library.