SigLIP 2: A better multilingual vision language encoder

Google's original SigLIP swapped CLIP's contrastive loss for a sigmoid loss and quietly became one of the more useful vision encoders around. SigLIP 2 keeps that foundation but bolts on a pile of new training tricks, and the result is a family of models that beats the old SigLIP at every single size, from the base checkpoint up to a new 1-billion-parameter giant variant.

The biggest addition is a text decoder sitting alongside the usual image and text encoders. During training it's tasked with three things: writing a full caption for an image, predicting bounding boxes from region descriptions, and doing the reverse — describing what's inside a given box. None of this is used at inference time, but forcing the encoder to support it teaches the model where things are in an image, not just what's in it broadly.

Google also borrowed a page from self-supervised learning with a teacher-student setup. The student sees only a cropped, local patch of an image and has to match the representation the teacher produces from viewing the whole thing. Separately, half the patches get masked out and the student has to reconstruct the teacher's features for those hidden regions. Both losses only kick in after 80 percent of training is done, mostly to keep compute costs sane, and the payoff is sharper, more spatially aware local features.

Then there's the resolution problem, which has quietly annoyed anyone building document or OCR pipelines with fixed-input vision models. SigLIP 2 ships two answers: a straightforward fixed-resolution path where positional and patch embeddings get resized and fine-tuned, and a more clever