Multimodal Evaluation
Training is half the loop. The other half is measurement. This lesson builds three evaluation surfaces from primitives: image-caption retrieval reported as R@1, R@5, R@10; visual question answering reported as exact match accuracy; and image captioning reported as BLEU-4. Each metric is a function over the model's outputs and a synthetic eval suite that runs in seconds.
90 minutesBuildPythonMembers
Learning Objectives
- Compute Recall@K from a similarity matrix between image and caption embeddings.
- Compute exact-match VQA accuracy from a model that maps (image, question) pairs to a fixed answer vocabulary.
- Compute BLEU-4 from generated and reference token sequences without any external library.
- Run all three evals against a synthetic suite built on top of the trained model from lesson 62.