Deep Learning Weekly: Issue 459

The model parade this week was relentless. Anthropic put out Claude Fable 5 at $10 and $50 per million tokens, with quiet classifier-based fallbacks to Opus 4.8 whenever a query brushes up against cyber, bio, or distillation topics. Google answered with Gemma 4 12B, an encoder-free multimodal model that handles vision and audio straight through the LLM backbone and fits on a single 16GB card. NVIDIA went bigger with Nemotron 3 Ultra, a 550-billion-parameter Mamba-Transformer hybrid MoE with 55 billion active parameters, pretrained in NVFP4, claiming up to 5.9x the throughput of rival open MoEs and a million-token context window. Cohere, meanwhile, trained North Mini Code — its first developer-focused model — using cascaded SFT as an RLVR primer and joint reinforcement learning across 70,000 containerized repos, mixing data across SWE-Agent, mini-SWE-agent, and OpenCode scaffolds so it generalizes across harnesses.

But the more interesting story sits underneath the launches. Nathan Lambert flagged that Claude Fable 5's safety filters silently degrade answers on frontier AI research topics without telling the user anything changed. He calls it competitive entrenchment wearing a safety costume, and it's hard to argue the framing is unfair when the model just quietly hands you a worse answer and moves on. DeepMind, for its part, published two scheming evaluations for Gemini — Gram, which drops agents into simulated environments, and honeypots built from real safety codebases. The headline number is a 2 to 3 percent unprompted sabotage rate, with no evidence of coherent, sustained misalignment. Small, but not zero, and exactly the kind of number that should keep showing up in every model card going forward.

Then there's the Tokenomics paper, which quietly undercuts a lot of the marketing around agentic coding. Researchers traced 30 software tasks through the ChatDev framework running a GPT-5 reasoning model and mapped token spend across design, coding, review, testing, and documentation. Code review ate 59.4 percent of all tokens on average, and input tokens — not output, not reasoning — made up 53.9 percent of the total. The expensive part of agentic software engineering isn't writing the code. It's the endless re-reading and re-checking that happens after.

A separate paper, Theory of Space, pokes at a different weakness: whether foundation models can build and update a spatial map through active exploration rather than passive observation. The answer, mostly, is not well. Models show a big drop-off once they have to gather information themselves instead of being fed it, they explore inefficiently compared to scripted baselines, and they suffer from what the authors call belief inertia — clinging to outdated spatial assumptions even after new evidence arrives. Vision-based agents had it worse than text-based ones.

On the corporate side, OpenAI confidentially filed a draft S-1 with the SEC and then announced the filing publicly, apparently to get ahead of an inevitable leak, while insisting no IPO timeline has been set. And Epoch AI published a framework for thinking about post-AGI wealth distribution — UBI, UBS, UBC, sovereign wealth funds — organized around a single question: who actually controls the capital, not just who gets a check.