MolmoPoint: Better pointing architecture for vision-language models

Allen Institute (AI2) · 3 months ago

Researchers at AI2 released MolmoPoint, a vision-language model architecture that uses special grounding tokens to enable pointing at locations in images and videos instead of generating text coordinates. MolmoPoint-8B achieves 70.7% accuracy on PointBench and 89.2 F1 on PixMo-Points, with three model variants released alongside new datasets including MolmoPoint-GUISyn containing 36,000 synthetic screenshots with 2 million annotated points. The grounding token approach reduces pointing from 8 tokens to 3 tokens, trains faster, and maintains accuracy across different image resolutions better than coordinate-based methods.