SportD: Can VLMs Physically Strategize?

arXiv cs.AI · 6 hours ago

Researchers introduced SportD, a benchmark of 478 soccer decisions from the 2022 FIFA World Cup, to evaluate whether vision-language models can make strategically sound decisions by choosing between shooting or passing. The best-performing VLM selected the highest-valued action 31.4% of the time compared to 38.9% for professional players, with models systematically preferring lower-risk passes and shooting less frequently than optimal. The models appear to partially imitate familiar play patterns rather than consistently evaluate different strategic options, suggesting current VLMs struggle with physical strategic reasoning.