Vision Encoder Patches
A vision model that reads pixels needs a tokenizer for pixels. Patch embedding is that tokenizer. Cut the image into a grid of squares, flatten each square, project it through one linear layer, then add a 2D position signal so the transformer knows where each square sat in the original image.
90 minutesBuildPythonMembers
Learning Objectives
- Tokenize an image into a fixed-length sequence of patch embeddings.
- Implement a
Conv2d-based patch projection that matches the math of unfold-then-linear. - Build a deterministic 2D sinusoidal position embedding so token order encodes spatial position.
- Verify patch count, embedding shape, and
Conv2d/unfold equivalence on a synthetic fixture.