ByteDance is training an AI model with up to 10 trillion parameters to compete with advanced US systems like Anthropic's offerings. The model is currently in pre-training, which typically takes three to six months, and would be three times larger than China's previous largest released model, Moonshot's Kimi K3. If successful, this would demonstrate that Chinese AI labs are closing the gap with leading US competitors in model scale and capability.
Researchers scaled Categorical Flow Maps (CFMs), a continuous flow matching approach for language models, to 1.7 billion parameters trained on 2.1 trillion tokens, demonstrating that the method can generate coherent text in as few as 4 inference steps while maintaining quality comparable to discrete approaches. The model maintains near-data-level token entropy and achieves results on standard benchmarks in the same range as discrete diffusion methods. This work establishes CFMs as a viable alternative to autoregressive language models at production scale, with detailed insights into loss weighting and scheduling challenges.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
