Can AI Agents Learn From Expert Corrections?
YouTube
OpenAI built a tax AI that learns from accountants fixing its mistakes, turning corrections into training data. The twist: it doesn't ask experts to just trust it blindly, it shows its work and gets graded.
Based on reporting by YouTube — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
OpenAI engineers John de Wasseige and Arthur Fernandes Araujo have been quietly running an experiment with something called Tax AI, and the details they've shared reveal a much more interesting problem than "can an agent do taxes." The real question is whether an agent can get better at doing taxes by watching professionals correct it, over and over, without turning the whole process into a black box nobody trusts.
Here's the mechanic. When an accountant fixes something Tax AI got wrong, that correction doesn't just vanish into a support ticket. It gets converted into a structured signal, essentially a labeled example of what right looks like versus what the model produced. Those signals feed into traces, which are basically replayable records of how the agent reasoned through a task, step by step. Stack enough traces together and you get evals, which let engineers test whether a fix in one area actually holds up, or whether it quietly breaks something else three steps downstream.
What's notable is the restraint built into this loop. De Wasseige and Fernandes Araujo describe scoped product fixes, meaning the team doesn't retrain the whole model every time an accountant flags an error. They patch the specific behavior, verify it against the evals, and move on. That's a very different philosophy from the usual AI narrative of bigger models solving everything through scale. This is closer to how a junior associate gets managed at an accounting firm: mistakes get corrected, patterns get noticed, and the corrections accumulate into something resembling judgment.
The bigger point, and probably the one OpenAI wants people to sit with, is that none of this requires accountants to take the agent's output on faith. The system is built so trust gets earned incrementally through visible correction, not demanded upfront through marketing. For an industry where a single wrong number can trigger an audit, that distinction between confidence and reliability is not a nuance. It's the whole game.
My take — AI-written commentary, not fact-checked reporting
This is the unglamorous, correct way to deploy AI in high-stakes work, and I wish more of the industry talked like this instead of chasing benchmark headlines. Tax prep is exactly the kind of domain where
Read more about this at: YouTube