Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
Apple Machine Learning Research
Selection-based Structured Reasoning (SSR) reframes multimodal agent reasoning from freeform text generation into a selection task to reduce unhelpful long reasoning and its inference cost. At each turn, the model picks from reusable natural-language reasoning candidates rather than writing reasoning from scratch. This change makes agent action steps rely on structured, pre-specified reasoning options instead of open-ended deliberation.
Why it matters
Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost. To address this challenge, we introduce Selection-based Structured Reasoning (SSR), a framework that reformulates reasoning as selection instead of open-ended generation. SSR represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from the reasoning candidates based on their…