Architecture Innovation

32 summarised stories about Architecture Innovation, each linking back to the original source. Browse all topics →

Wednesday, 17 June 2026

The Sequence AI of the Week #878: Inside Google Deepmind's First Real Crack in Next-Token Generation

TheSequence 4 weeks ago

Google DeepMind released DiffusionGemma, a text-diffusion model that generates text differently from transformer-based language models by not sequentially predicting one token at a time. The model uses diffusion-based generation to challenge the conventional left-to-right, next-token prediction approach that has dominated the modern LLM era. This alternative architecture potentially offers a different pathway for text generation beyond the sequential token-by-token method used in GPT-style models and other transformer-based systems.

Algorithm Innovation Architecture Innovation Google DeepMind

GLM-5.2: Built for Long-Horizon Tasks

Hugging Face Blog 4 weeks ago

Zhipu released GLM-5.2, a language model designed for long-horizon coding tasks with a stable 1M-token context window. The model achieves 81.0 on Terminal-Bench 2.1 (compared to 63.5 for GLM-5.1) and ranks as the highest open-source model on three long-horizon coding benchmarks. The architecture introduces IndexShare, which reduces per-token computation by 2.9× at 1M context length, and users can now select effort levels to balance performance against latency and computational cost.

Architecture Innovation Large Language Models Open Source