SkillOpt: Agent skills as trainable parameters

Anyone who's spent time building AI agents knows the dirty secret: the instructions you give them, often called "skills," get tweaked by trial and error. Someone notices the agent fails a task, edits the prompt, hopes for the best, moves on. There's no held-out test set, no rollback plan, no way to know if the fix actually helped or just papered over one bug while creating three others. Microsoft Research's new system, SkillOpt, treats that whole mess as a solvable engineering problem rather than a vibe check.

The core idea is almost mundane in its simplicity: freeze the underlying model, and instead train the skill file sitting outside it. SkillOpt runs a forward-backward-update loop that looks a lot like classic machine learning, minus the gradients. The frozen model executes tasks, a separate optimizer model reads the resulting transcripts and proposes small add/delete/replace edits, and every candidate skill has to clear a validation gate — it only survives if it beats the current version on held-out data. Rejected edits don't get thrown away either; they become negative examples that steer future proposals away from repeating the same mistake.

The results are hard to wave off. Across six benchmarks, seven models ranging from GPT-5.5 down to the 4-billion-parameter open-weight Qwen3.5-4B, and three different execution setups, SkillOpt came out best or tied-best in all 52 test combinations. On GPT-5.5 alone, average scores jumped from 58.8 to 82.3 — a 23.5 point swing without touching a single model weight. Spreadsheet tasks nearly doubled, from 41.8 to 80.7. And the gains weren't confined to chat: plugging the same approach into Codex and Claude Code agent loops still added 20-plus points over no skill at all.

What's more interesting than the raw numbers is what the optimized skill files look like. They're small — a median of about 920 tokens — and built from just one to four accepted edits, since the validation gate rejects most proposals outright. One OfficeQA skill jumped performance by 39 points from a single accepted edit. That compactness matters because it means the resulting text is readable and auditable, not a bloated pile of accumulated patches nobody can review anymore.

The transfer results are maybe the strongest evidence that something real is being learned here, rather than benchmark memorization. A spreadsheet skill trained inside Codex, dropped unmodified into Claude Code, lifted that harness's baseline from 22.1 to 81.8 — matching or beating a skill trained natively inside Claude Code. Smaller models benefited too: GPT-5.4-mini with an optimized skill outperformed the larger, skill-less GPT-5.4, and Qwen3.5-4B cleared GPT-5.2's baseline. If a cheap open-weight model with the right skill file can match a pricier closed model with none, that's a genuinely useful lever for anyone trying to control agent costs.