This could be the largest synthetic code dataset yet

IBM Research just released something that sounds dry on paper but could quietly reshape how coding AI models get trained: a nearly 1-trillion-token synthetic dataset called CodeAlchemy, built to fix a problem most people outside AI labs never think about. Good code on the internet is getting scarce. Either it's already been hoovered up by existing models or it's locked behind commercial licenses, leaving mostly mediocre code freely available for anyone building the next generation of tools.

The pipeline itself started as an internal project to improve IBM's Granite models, but the company is now open-sourcing both the tooling and the dataset. Covering 15 programming languages, it's reportedly at least 200 times the size of Wikipedia's roughly 5 billion tokens. What sets it apart isn't just scale, though. Included in the release are 1.3 million code files paired with their actual execution traces, meaning researchers didn't just dump code — they ran it and recorded what happened.

That distinction matters more than it sounds. IBM researcher Ankit Gupta put it plainly: pulling billions of lines from GitHub teaches a model what code looks like, not what it does when you actually run it. A developer tracing a program line by line can catch bugs, spot performance issues, and flag security risks. CodeAlchemy is, by IBM's account, the first dataset attempting to teach models that same skill by pairing static code with real runtime behavior.

The results on frontier models were humbling. When researchers tested how well models could predict a line of code purely from its trace, Claude Sonnet 4.5 nailed an exact match less than 6% of the time, and got a partial match only about a third of the time. Smaller models fared even worse on exact matches. But after training an early Granite 4.0 3B model on CodeAlchemy data, it improved on partial matches enough to beat larger models, and its win rate against Claude Sonnet on a benchmark called DevEval jumped from 2% to 8%.

Some of the findings ran against intuition. Using a smaller model, Gemma 4B, to rewrite code for training produced better outcomes than using a larger GPT-OSS 20B model, apparently because bigger models generalize toward sameness while synthetic data needs variety to be useful. IBM also found that 100 billion tokens of filtered synthetic code outperformed 600 billion tokens of real code on the same model. A Granite 4.0 3B model trained on the CodeAlchemy mix and fine-tuned afterward scored 83.5% on HumanEval and 63.2% on MBPP, suggesting the gains stick around even after further training.

IBM isn't stopping at code. The same synthetic-data approach is being extended into a broader effort called Open Alchemy, aimed at teaching AI agents to operate in specialized work environments — calling applications, navigating databases, handling file systems. The CodeAlchemy team is also building out multi-step coding tasks covering debugging and dependency management, betting that as coding agents take on more autonomous work, they'll need training data that looks less like a GitHub scrape and more like an actual job.