Scaling AI-driven chip design with open agent factories

IBM Research is arguing that AI has moved past code suggestions and into the messier business of hardware design. That is a bigger claim than it sounds like. Writing a chip is one thing; deciding how to organize computation, move data, and squeeze useful work out of limited hardware is another, and those choices multiply fast.

The company’s answer is an “agent factory”: a coordinator sets the search, several worker agents try different hardware ideas in parallel, a verifier checks the results, and synthesis feedback tells the next round what to try. In the new study, the team used four worker agents and gave each configuration a budget of 100 hypotheses. The target was Relay-BP, a quantum error correction decoder for the [[144,12,12]] bivariate bicycle code.

The workload is a good fit for this kind of experiment because it already contains parallel message passing, but the baseline implementation hides that structure from high-level synthesis tools. So the agents were asked to change the decoder without breaking it. Some of the useful moves were architectural, not cosmetic: reorganizing message storage so independent memory accesses became visible, or replacing sequential reductions with tree structures that expose parallel work.

IBM compared three setups under the same search budget: all proprietary frontier models, all open frontier models, and a hybrid with a proprietary coordinator and open-model workers. All three produced verified improvements. The all-open setup reached a 52x speedup in HLS estimates at about $26 in inference cost, while the proprietary setup reached 204x but cost $299. The hybrid sat between them. The open run also used more tokens, about 241 million versus 216 million for the all-proprietary search, but its serving cost was far lower.

That gap came from the plumbing as much as the model choice. IBM ran the open models on premises through llm-d, which used cache-aware routing, separated prompt processing from token generation, and reportedly hit a prefix-cache rate of about 98% with a blended serving cost of roughly $0.11 per million tokens. In the same setup, pushing the open search from 100 to 200 hypotheses lifted speedup from 52x to 60x, which is a gain, just not the kind that makes budget people throw confetti. The point is simpler: cheaper inference buys more verified exploration, and more exploration is what hardware design actually needs.