Plan or Retrieve First? A Scale Study of Tool Discovery Architectures
Apple Machine Learning Research
Tool-augmented LLM agents benchmark two tool-discovery architectures—Retrieve-then-Plan and Plan-then-Retrieve—on ToolBench to see which finds the right APIs before execution. PtR beats RtP on multi-tool queries across retrieval budgets k=1 to 20 and registry sizes from 1k to 47k APIs. The results imply complexity-adaptive routing (potentially using RtP’s top-1 retrieval score) to choose between the two pipelines.
Why it matters
Tool-augmented LLM agents discover relevant APIs from large registries before planning execution, yet finite context windows prohibit injecting all tool definitions into a single prompt. The dominant approach, which we term Retrieve-then-Plan (RtP), embeds the user query and retrieves similar tools via approximate nearest-neighbor search. We study an alternative Plan-then-Retrieve (PtR) paradigm in which the LLM first generates Abstract Tool Stubs (hypothesized function signatures describing needed capabilities) which then serve as retrieval queries against the registry. Evaluating on…