Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese

GitHub Pages · 3 years ago · 46

Researchers developed Chinese CLIP, a vision-language model adapted for Chinese by using a two-stage pretraining method with frozen and unfrozen encoders on 200 million image-text pairs from public datasets. The model achieved the best performance on Chinese cross-modal retrieval benchmarks like MUGE, with notably larger improvements over previous models on native Chinese datasets than on English-native datasets. This language-specific approach addresses limitations of directly translating English CLIP outputs and enables better understanding of Chinese cultural concepts in vision-language applications.