Just Keep Prompting: Evaluating Repetitive Socratic Prompting in VLMs

arXiv cs.CL · 6 hours ago

Researchers evaluated Vision-Language Models using a multi-turn framework called Just Keep Prompting that repeatedly challenges model answers through adversarial negation and Socratic questioning to measure epistemic stability. Testing GPT-4o, Gemini 2.5 Pro, and Qwen3-VL-30B across 720 multi-turn runs showed that while aggregate accuracy changed modestly from Turn 0 to Turn 10, trajectory-level analysis revealed substantial instability with correct answers regressing and wrong answers recovering. The findings demonstrate that repeated prompting often destabilizes rather than aids reasoning, with effects strongly model-dependent, revealing how VLMs trade off visual grounding, calibration, and conversational compliance under sustained challenge.