Get Started

Open-Vocabulary Vision, CLIP

Train an image encoder and a text encoder together so that matching (image, caption) pairs land at the same point in a shared space. That is the whole trick.

45 minutesBuild + UsePythonMembers

Learning Objectives

  • Explain CLIP's two-tower architecture and contrastive training objective
  • Use a pretrained CLIP (or SigLIP) for zero-shot classification without any task-specific training
  • Implement zero-shot classification from scratch: encode class prompts, compute cosine similarity, take argmax
  • Distinguish CLIP, SigLIP, OpenCLIP, and LLaVA/LLaMA-vision models, what each is for in 2026