Algorithm Innovation

108 summarised stories about Algorithm Innovation, each linking back to the original source. Browse all topics →

Saturday, 16 May 2026

Latest open artifacts (#21): Open model bonanza! Gemma 4, DeepSeek V4, Kimi K2.6, MiMo 2.5, GLM-5.1 & others. On CAISI's V4 assessment.

Interconnects 2 months ago

Multiple open-source AI labs released new frontier models this month, including DeepSeek V4, Gemma 4, Kimi K2.6, and others, prompting a fresh evaluation of how open models compare to closed alternatives. The Center for AI Standards and Innovation found open models lag by 3-7 months behind American frontier models, with the gap widening when measured against proprietary benchmarks like CTF-Archive-Diamond and PortBench. The benchmarking methodology itself remains contested, as standardized evaluation setups do not match how models are actually deployed—for example, using basic coding harnesses rather than the specialized environments models are trained for.

Algorithm Innovation Benchmarking Foundation Models Open Source

Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention

Ahead of AI 2 months ago

Recent open-weight LLM releases including Gemma 4, DeepSeek V4, and others have adopted architectural techniques like KV sharing across layers, per-layer embeddings, and compressed attention to reduce memory and compute costs for long-context processing. Gemma 4 E2B achieves approximately 2.7 GB of KV cache savings at 128K context length through cross-layer KV sharing that allows later transformer layers to reuse key-value tensors from earlier layers. These efficiency-focused design changes enable smaller models to handle longer contexts and reduce memory requirements, which becomes critical as reasoning models and agent workflows maintain more tokens during inference.

AI Infrastructure Algorithm Innovation Architecture Innovation