CLIP: Connecting text and images
OpenAI Blog
OpenAI introduced CLIP, a neural network that learns visual concepts by connecting text descriptions with images. CLIP performs zero-shot classification on visual benchmarks without requiring task-specific training data. This approach enables the model to recognize new image categories by simply receiving their text names, reducing the need for large labeled datasets in computer vision tasks.
Why it matters
We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized, similar to the “zero-shot” capabilities of GPT-2 and GPT-3.