ALTK-Evolve: How learned memory tops the τ²-bench leaderboard on customer service tasks without a frontier model
Hugging Face
ALTK-Evolve taught a customer-service bot a better playbook from its own runs. That pushed a stock GPT-4.1 agent to the top of τ²-bench retail without changing model weights.
Based on reporting by Hugging Face — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
A prompt can still beat fine-tuning. Hugging Face’s ALTK-Evolve setup takes GPT-4.1, looks at how the agent handled training tasks, and turns those traces into short rules that get pasted back into the system prompt. No weight updates. No human labeling. Just learned memory, bolted onto a stock ReAct agent.
On τ²-bench, a customer-service benchmark built around policy-heavy tasks, that matters because the agent has to do more than chat politely. It has to pull the right details from a simulated customer, follow a written policy, use the right tool, and sometimes make irreversible changes like refunds or cancellations. Success is strict: the final database state has to match the expected outcome, and the benchmark scores the agent on Pass^4, meaning it has to solve a task in all four runs.
With GPT-4.1 and no learned guidelines, the retail agent already reached 67.5% Pass^4 on the benchmark’s base split. But the full ALTK-Evolve playbook pushed performance higher on the held-out retail tasks too: mean accuracy rose from 78.1% to 82.5%, and Pass^4 climbed from 57.5% to 67.5%. Airline improved as well, though the best guidance mix differed there. The compact high-support set helped reliability more cleanly than the full set, while the full set still lifted accuracy.
The cost tradeoff is boring in the best way. Better results came from longer prompts, not longer conversations. In matched solved runs, the guided agent actually used slightly fewer assistant turns, which trims latency as well as cost. The extra spend showed up in prompt tokens, and the blog notes that prompt caching can recover a lot of that in production.
And the approach didn’t stop at an older model. When Hugging Face reran airline with GPT-5.5 for both the agent and the guideline miner, the same method still improved the system: accuracy rose and Pass^4 hit 70.0% on the held-out airline tasks. On the published τ²-bench leaderboard snapshot from October 12, 2026, that meant ALTK-Evolve reached the top of retail with GPT-4.1 and the top of airline with GPT-5.5, all without touching model weights.
My take — AI-written commentary, not fact-checked reporting
This is the annoying truth for people who want every win to come from a bigger model: a disciplined prompt can still do serious work. The industry keeps treating memory like a garnish, then acts surprised when it behaves like infrastructure. Fine-tuning gets the headlines; prompt engineering keeps collecting the receipts.
Read more about this at: Hugging Face