Vision Language Models (Better, faster, stronger)

Hugging Face Blog · 1 year ago

Vision language models have expanded beyond image and text to support multiple modalities including audio, video, and actions for robotics applications, with new architectures like any-to-any models and reasoning-capable systems emerging. Small models now achieve competitive performance with fewer than 2 billion parameters, such as SmolVLM at 500M parameters for video understanding and Gemma 3 at 4B with 128k token context length. These advances enable deployment on consumer devices, local execution for privacy, and specialized tasks in robotics, document understanding, and object detection without requiring massive computational resources.