Google Research Releases ToolGrad: Answer-First Framework Hits 99.8% Pass Rate for Tool-Use Data Generation
MarkTechPost Michal Sutter ● Covered by 2 sources
Google researchers flipped tool-data generation: build the API chain first, then write the user query. That pushed pass rate to 99.8% and got a 12B Gemma-3 model near frontier closed models.
Based on reporting by MarkTechPost, Michal Sutter — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Training a model to use tools is only as good as the data behind it, and that data has been expensive to make. Google researchers, working with the University of Tokyo, RIKEN AIP, and Tohoku University, say they found a cleaner way: stop guessing the prompt first and verify the tool chain first.
Their method, ToolGrad, turns the usual pipeline inside out. Instead of asking a model to invent a user request and then hunting for a path that fits, it starts by actually executing APIs until it has a working chain. Only then does it ask a model to write a matching query and response. That matters because a real chain is concrete. A hypothetical prompt is slippery, and slippery data burns compute.
The system runs in four steps. An API Proposer narrows the field, API Executors try the options in parallel, API Selector picks the best call and adds it to the workflow, and LLM Updater rewrites the synthetic query so it matches the new set of APIs. The paper describes the feedback from execution reports as a directional signal — a sort of textual gradient. In the default setup, the loop runs 10 iterations over 50 sampled APIs per workflow.
On ToolBench, which has more than 16,000 real-world APIs, ToolGrad sharply outperformed the older DFS-based query-first approach. The pass rate rose from 63.8% to 99.8%. ToolGrad also produced longer chains, with ground-truth tool uses per sample rising from 2.1 to 3.4, while tool-use steps per sample fell from 34.3 to 20.0. LLM calls barely changed, sliding from 64.5 to 63.9. The only failure case came when the agent couldn’t get a successful response from three selected APIs across all 10 iterations and saved an empty sample.
The team also used a 500-sample set, ToolGrad-500, to post-train Gemma-3 models at 1B, 4B, and 12B parameters. On the Berkeley Function Calling Leaderboard, the 12B model scored 83.1, just behind Gemini 2.5 Pro at 83.2 and ahead of Claude 4.5 Opus at 82.8 and GPT-5 at 74.4, as measured at publication. The same 12B student also beat the Gemini 2.5 Flash-Lite model that generated its training data. The code is Apache-2.0, and the dataset plus models are on Hugging Face, with a PyPI package too.
My take — AI-written commentary, not fact-checked reporting
This is the rare AI paper that sounds less like hype and more like workflow hygiene. If the chain is what matters, building the chain first is obviously the grown-up move. The industry keeps pretending prompt invention is the hard part when, half the time, it’s just a noisy way to waste compute.
Read more about this at: MarkTechPost