Hugging Face Blog
Researchers trained CodeParrot, a GPT-2 model for Python code generation, from scratch using a cleaned dataset of GitHub code. The training dataset comprised 50 GB of Python code after removing duplicates that initially made up 66% of files in the original 180 GB collection. The model uses streaming data loading and distributed training across multiple GPUs to enable practical training without downloading the full dataset to disk.
CoderForge-Preview: SOTA open dataset for training efficient coding agents
Together AI · 4 months ago
Data Machina #258
Data Machina · 2 years ago
The daily briefing
TLDRocket reads 60+ sources, removes duplicate coverage, and summarises the day in two minutes. Free, no spam, unsubscribe anytime.
TLDRocket
21 July 2026
The open-weight AI model market hit an inflection point today as Alibaba and Kimi released trillion-parameter competitors that challenge the closed-model orthodoxy—just as Washington moved to restrict them. Alibaba's Qwen 3.8 Max (2.4 trillion parameters) and Kimi K3 (2.8T) are going open-weight, with K3 ranking first on Frontend Web App Arena and fourth on agentic benchmarks, signaling that scale and transparency can coexist. The timing matters: while U.S. policymakers weigh layered restrictions on Chinese open models, technical experts are pushing back hard, arguing that closed gardens actually hurt security and competition more than they help. The real fragmentation risk isn't technical—it's regulatory.
Read the full briefing →