Microsoft Open Sources code-testing-generator: a Polyglot Unit-Test Agent That Hits 92.1% Task Completion Versus 78.9% for Stock Copilot
MarkTechPost Michal Sutter
Microsoft open-sourced a tool that writes unit tests, runs them, and checks its own work before calling it done. It beat Copilot 92% to 79% on the same benchmark, mostly by handling vague prompts better.
Microsoft's .NET team just dropped code-testing-generator into the open, and it's aimed at a problem every developer knows: asking a coding assistant to "write some tests" and getting back something that compiles but tells you nothing useful. The tool lives inside the MIT-licensed dotnet/skills repository as the dotnet-test plugin, and instead of guessing at frameworks and file paths, it actually reads the repository first.
The design follows what Microsoft calls a Research-Plan-Implement pipeline. Before writing a single line, the agent scans the codebase, figures out which language and test framework are already in use, studies existing tests for house style, and — crucially — tracks down the real build and test commands. That last part matters more than it sounds. Plenty of test suites pass locally and then vanish into a CI pipeline that never actually runs them, because nobody wired them in. The agent picks one of three modes depending on scope: a quick direct pass for small jobs, a single validated cycle, or an iterative loop for bigger coverage pushes. It won't touch production code, and it steers clear of flaky test patterns like hitting live URLs or timing-dependent assertions.
Before declaring victory, it runs itself through five checks, including a lightweight mutation-testing step where it imagines small code changes and confirms its tests would catch them. It also verifies that every requested scenario actually has a corresponding test, builds the whole workspace, runs the full suite, and makes sure the repo's own test runner picks up the new files.
The numbers back up the pitch. On Microsoft's internal benchmark of 152 real-repo tasks, the agent finished 140 (92.1%) against 120 (78.9%) for plain GitHub Copilot using the identical model and prompts — a 63% cut in failures. The gap is almost entirely explained by vague prompts: 88.8% completion versus 66.3% when instructions were sparse, while both tools tied at 96.8% on detailed prompts. On 15 tasks tied to a specific pull-request diff, the agent went 15-for-15 while stock Copilot went 0-for-15. It also wrote 2.3% fewer tests for essentially the same line coverage, and finished about 5.5% faster.
This isn't a hosted service — it's an agent definition that plugs into whatever coding assistant you're already running, so code never leaves your machine. That makes it immediately usable by a solo maintainer or a three-person startup that has no time to reverse-engineer its own testing conventions, while larger, audit-heavy shops in finance, healthcare or the public sector get a fork-friendly base for enforcing internal standards across sprawling polyglot codebases.
My take
The real story here isn't the 13-point completion bump, it's that Copilot-style one-shot generation basically falls apart the moment a prompt is vague or scoped to a diff, which is most real-world usage. Bolting a research-and-verification loop onto an existing model, rather than shipping another hosted black box, is the more honest way to fix an agent's blind spots, and it's a pattern the rest of the industry keeps ignoring in favor of bigger models instead of better scaffolding.
Read more about this at: MarkTechPost
