Get Started

Self-Supervised Vision, SimCLR, DINO, MAE

Labels are the bottleneck of supervised vision. Self-supervised pretraining removes them: learn visual features from 100M unlabelled images, fine-tune on 10k labelled ones.

75 minutesLearn + BuildPythonMembers

Learning Objectives

  • Trace the three major self-supervised families, contrastive (SimCLR), teacher-student (DINO), masked reconstruction (MAE), and state what each one optimises
  • Implement an InfoNCE loss from scratch and explain why a batch of 512 works but a batch of 32 fails
  • Explain why MAE's 75% masking ratio is not arbitrary and how it differs from BERT's 15% for text
  • Use DINOv2 or MAE ImageNet checkpoints for linear probing and zero-shot retrieval