Agentic Coding Is Straining CI. Here's How We Scaled Test Impact Analysis at Anthropic
Claude ● Covered by 2 sources
Anthropic says Claude now writes most of its code, and CI is feeling it. Their test service had to be rebuilt after quick fixes stopped holding.
Based on reporting by Claude — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Anthropic’s engineers are shipping far more code than they used to, and the company says Claude is writing 80% of it. That changes the bottleneck. Code generation is no longer the hard part. Review is speeding up too, and CI is left to absorb the blast radius.
The numbers in the post are hard to ignore. Code shipped per quarter is up 8x on average compared with 2021-2025, the test count in the codebase is up 10x, and CI jobs jumped 25x in six months. Anthropic says it added only a nominal number of engineers. So the workload on the test impact analysis service kept climbing while the human headcount barely moved.
That service had a simple job: keep a listener and a selector in sync. The listener recorded test results from CI runs; the selector used that history to decide which tests should run on a given PR. But the design depended on a single writer, which meant it could not be split across machines. When CI traffic rose, listener lag started to pile up behind the PR queue, and even 20 minutes of lag could leave tens of thousands of test updates unprocessed.
Anthropic tried three quick fixes before giving up on patching the old shape. A bigger machine bought time for 70 days. Sharding per package bought 29 days. Daily restarts bought less than a day, and sometimes made the backlog worse. By March, the process was hitting memory limits by mid-afternoon on weekdays, and the service was still falling behind often enough to leave stale test-selection data in place.
The rebuild changed the architecture instead of trying to nurse the singleton along. The new setup gives the service an in-memory data store, lets any listener worker append results to a journal, and uses a separate consumer to roll that journal into per-test history every few seconds. Anthropic says the project took three weeks for one engineer, while a year earlier it would have taken closer to a quarter. Since the cutover, the backlog has stayed flat and the service has remained stable.
My take — AI-written commentary, not fact-checked reporting
This is the part of agentic coding nobody sells as a feature: the code gets cheaper, the plumbing gets meaner. Teams that keep pretending CI will stay quaint are going to learn the hard way that singletons are a hobby, not a strategy. The boring closed-loop stuff now matters more than the flashy model demo, which is exactly how real infrastructure always wins the argument.
Read more about this at: Claude