Hugging Face Releases Carbon-X, a 14B Open DNA Language Model that Brings DNA and Natural Language Together for the First Time

Hugging Face has released Carbon-X, a 14.8 billion-parameter DNA language model from Hugging Face Bio. The pitch is simple enough to fit in a chat window: feed it raw DNA, ask it questions in English, and it responds with predictions about mutations, regulatory regions, organisms, and next bases.

Carbon-X is a decoder-only model built on Qwen3-14B, but it does not bolt DNA onto a separate encoder. Instead, text and DNA share one token stream inside one decoder. The interesting bit is the tokenizer. Hugging Face kept Qwen3’s BPE vocabulary for text, then added 5,515 DNA tokens, including 4,096 non-overlapping 6-mers, 1,364 tail tokens and 3 markers. One DNA token covers 6 base pairs, and stretches of 6 or more A/C/G/T bases are tagged automatically.

The model card says training happened in four stages. First came about 10 billion tokens of embedding warmup with the transformer frozen. Then came 302 billion tokens of mid-training at sequence length 8,192, with a mix of 55% raw DNA, 20% English, 15% biology text and 10% instruction data. After that, the team ran supervised fine-tuning over 20 dataset subsets, with deterministic verifiers checking responses, and then two rounds of asynchronous GRPO reinforcement learning with verifiable rewards.

Hugging Face says the hardest part was the learning rate. Push it too high and the model learns DNA but forgets English. Pull it back, and the balance holds. The company also says direct reinforcement learning on Qwen3-14B never beat a 50% chance baseline on FrontierVEP, which makes Carbon-X look less like a neat demo and more like the result of a lot of ugly training trade-offs.

On benchmarks, the numbers are self-reported and Carbon-X ran with thinking disabled, while competitors used their best reasoning effort. Hugging Face puts its average at 66.7% across five DNA benchmarks, just behind GPT-6.1 Sol at 67.5%, and says Carbon-X leads every open model tested. It tops BioReason Coding at 74.9% and NextBase at 39.4%, but it only manages 29.8% on GenomeQA splice-site questions. That gap matters. This is not a finished biology oracle; it is a strong open model with very uneven spots.

The practical hook is cost and deployment. Hugging Face says Carbon-X can run on a single 80 GB GPU in bfloat16 through vLLM or Transformers, and its Q8_0 GGUF weighs 15.8 GB. On an H200 priced at $5 per hour, the company says inference works out to roughly 1/1,800 of GPT-6.1 Sol’s cost per problem, and screening 2.4 million ClinVar variants of uncertain significance would be about $300 instead of $32,000 to $4 million on frontier models. For labs that want to keep sensitive sequences off a third-party API, that is the real selling point.