This could be the largest synthetic code dataset yet
IBM Research
IBM open-sourced CodeAlchemy, a pipeline that generates synthetic code training data covering 15 programming languages, totaling nearly 1 trillion tokens—at least 200 times larger than Wikipedia. The dataset includes 1.3 million code files paired with execution traces, a novel approach to teach models what code does at runtime rather than just syntax. Models trained on CodeAlchemy showed measurable improvements: a Granite 3B model trained on the synthetic data outperformed larger models on code reasoning tasks and achieved 83.5% on HumanEval benchmarks.
Why it matters
Introducing CodeAlchemy, a synthetic data pipeline that has already produced nearly 1 trillion tokens of open-source code
