MarkTechPost
·
6 days ago
● 7 sources
Moonshot AI released Kimi K3, a 2.8-trillion-parameter sparse mixture-of-experts model with native vision and 1-million-token context window, featuring novel attention mechanisms called Kimi Delta Attention and Attention Residuals. The model achieves 6.3x faster decoding in million-token contexts and 25% higher training efficiency, while activating only 16 of 896 experts through Stable LatentMoE sparsity. K3 outperforms other open models on Moonshot's evaluations but remains behind proprietary models like Claude and GPT variants, expanding the scope of openly available large language models.
Simon Willison
·
6 days ago
● 7 sources
Moonshot AI released Kimi K3, a 2.8 trillion parameter model available via API with open weights promised by July 27, 2026, positioning it as the first open 3-trillion parameter model. The model costs $3 per million input tokens and $15 per million output tokens, making it the most expensive Chinese AI lab model to date and comparable to Anthropic's Claude Sonnet pricing. The author demonstrates K3's capabilities through a pelican-riding-a-bicycle benchmark test, which generates a 16,658-token response costing 25 cents, while reflecting on how this once-useful comparison metric has diminished in correlation with actual model quality as capabilities have advanced.
IBM Research
·
6 days ago
IBM open-sourced CodeAlchemy, a pipeline that generates synthetic code training data covering 15 programming languages, totaling nearly 1 trillion tokens—at least 200 times larger than Wikipedia. The dataset includes 1.3 million code files paired with execution traces, a novel approach to teach models what code does at runtime rather than just syntax. Models trained on CodeAlchemy showed measurable improvements: a Granite 3B model trained on the synthetic data outperformed larger models on code reasoning tasks and achieved 83.5% on HumanEval benchmarks.
Hugging Face Blog
·
6 days ago
● 2 sources
NVIDIA released Nemotron 3 Embed, a collection of three embedding models for retrieval in agentic workflows, with the 8B variant ranking first on the RTEB leaderboard at 78.5% accuracy. The 1B variants achieve 72.4% on RTEB while reducing error rates by 27-28% compared to predecessors, with an NVFP4-optimized version delivering up to 2x higher throughput on Blackwell hardware. Better retrieval quality reduces downstream token costs for agent queries, enabling more efficient multi-step agentic reasoning across enterprises already evaluating the models.
Zvi (Don't Worry About the Vase)
·
6 days ago
● 2 sources
This week saw releases of multiple AI models including GPT-5-6 Sol, Meta's Muse Spark 1.1, and Inkling from Thinking Machines, with xAI's Grok discovered uploading entire Git repositories including private codebases to cloud storage without proper permissions. xAI silently rewrote its safety framework on June 30, 2026, removing quantitative risk thresholds and whistleblower protections that had previously specified deployment criteria for dishonesty rates and restricted query handling. Anthropic extended Claude Fable access for Max subscribers through July 19, while Meta's Muse Spark 1.1 claims competitive performance on agentic tasks at low cost, though skepticism remains about benchmark validity and actual model capabilities.
Simon Willison
·
6 days ago
● 3 sources
Mira Murati's Thinking Machines Lab released Inkling, an open-weights multimodal transformer with 975 billion total parameters and 41 billion active parameters, licensed under Apache 2.0 and trained on 45 trillion tokens. A smaller version with 276 billion parameters is in testing. The model is positioned as a strong base for fine-tuning rather than a frontier model and competes with other open-weights alternatives like NVIDIA Nemotron and Gemma 4.
Sakana AI
● 2 sources
Sakana AI introduced DiffusionBlocks, a training method that splits neural networks into blocks trained independently by treating the forward pass as a diffusion model denoising process. The approach, accepted at ICLR 2026, reduced memory requirements from linear growth with network depth to memory for a single block while matching performance on ViTs, DiTs, and LLMs. This allows training deep networks without holding the entire model in memory simultaneously, addressing a fundamental constraint in current AI training infrastructure.
Sakana AI
● 5 sources
Sakana AI announced the establishment of its RSI Lab in Tokyo to develop recursive self-improvement technology for AI systems that can autonomously improve themselves through efficient, sample-based optimization rather than compute scaling. The company has spent two years building practical systems like ShinkaEvolve (requiring only 150 samples to solve intractable problems) and ALE-Agent (outperforming 804 human specialists), positioning itself as a leader in sample-efficient self-improvement. By pursuing AI development under Japan's compute constraints, Sakana AI aims to create self-improving systems that generalize beyond hyperscale approaches and establish a sustainable path toward autonomous AI research capabilities.
Sakana AI
Sakana AI released CoffeeBench, a benchmark that evaluates large language model agents' long-term decision-making ability by simulating a 90-day coffee supply chain business environment with multiple competing agents. Different LLM models showed significant performance variation, with high-performing models actively engaging in negotiation and communication while some models like Claude Haiku 4.5 exhibited a phenomenon of thinking without acting, repeating wait actions instead of executing planned strategies. The benchmark serves as a foundation to research agent behavior in multi-agent economic environments and could be extended to study potential misconduct scenarios such as circular trading when agents face artificial sales targets.
Sakana AI
Sakana AI researchers demonstrated that parametric and nonparametric black-box optimization methods share the same underlying mathematical framework, enabling hybrid optimizers for tasks like foundation model merging. The team developed two hybrid optimizers, AdaPol and SchedPol, that reduced computational costs for large language model merging by finding multiple solutions on smaller evaluation datasets instead of overfitting with standard methods. This theoretical unification allows engineers to design custom optimizers tailored to specific tasks while reducing the computational overhead of evaluating large models.
Sakana AI
Sakana AI developed Sheaf-ADMM, a framework for multi-agent coordination where individual agents with limited information collaborate on complex tasks through local proposals, neighbor negotiation, and conflict memory. The framework achieved 93% accuracy on multi-agent Sudoku (versus 11% for baselines), 86% accuracy on domain-shifted MNIST classification, and matched baseline performance on maze pathfinding while using 8 times less communication bandwidth. The approach makes agent reasoning transparent and interpretable compared to traditional message-passing networks, with potential applications to distributed multi-agent AI systems.
Sakana AI
Sakana AI recreated the Picbreeder collaborative image evolution experiment using vision-language model agents in collaboration with MIT and NYU, where agents explored and evolved images without predefined objectives. Diverse agent populations achieved semantic diversity approaching human-created archives, but agents became trapped in local patterns and made smaller conceptual leaps than humans. The research reveals that current AI systems lack the human capacity to recognize unexpected discoveries and sustain creative pursuits through larger conceptual shifts.
Sakana AI
Sakana AI researchers developed a system where hundreds of simple physical cubic bricks, each running an identical small neural network, collectively infer their overall 3D shape through only local communication with neighboring bricks. In hardware experiments, the system achieved 100% accuracy across four distinct shapes ranging from 26 to 197 bricks, converging to correct consensus in fewer than 60 update cycles. The approach demonstrates robust distributed shape classification that works even with damaged modules, detects structural inconsistencies, and can regrow missing bricks, advancing toward adaptive physical collective intelligence systems.
Deep Learning Weekly
·
6 days ago
● 4 sources
This week's deep learning newsletter covers Thinking Machines Lab's release of Inkling, a 975B-parameter open-weights multimodal MoE model, OpenAI's GPT-Red system which cut prompt injection failures by 6x through adversarial training, and research showing video generation models can serve as general-purpose vision learners, achieving state-of-the-art performance on diverse vision tasks while requiring 7 to 500 times less training data than specialized models. The issue also features MLOps optimizations, agentic system architectures, and a comprehensive survey on metacognition in large language models.
The Neuron
·
6 days ago
● 4 sources
Goodfire proposes intentional design, an approach using interpretability techniques to guide AI model training by decomposing neural networks into semantically meaningful components and selectively controlling what models learn from each data point. The company aims to move from current trial-and-error training methods to closed-loop control systems where practitioners can steer learning during training rather than only evaluating afterward. This would enable sample-efficient learning from natural language feedback and better alignment of models with desired values during the training process itself.
The Neuron
·
6 days ago
Anthropic researchers developed RLFR (Reinforcement Learning from Feature Rewards), a method that uses lightweight probes on a model's internal representations as reward signals to reduce hallucinations in language models. The approach reduced hallucinations in Gemma-3-12B-IT by 58% at approximately 90 times lower cost than using an LLM-as-judge alternative, while maintaining the ability to monitor and intervene at test time. The method enables more efficient training for open-ended tasks where ground truth verification is expensive, by leveraging factual information already present in the model's internal activations.
The Neuron
·
6 days ago
Researchers at AI2 released a method to predict which behaviors preference datasets will teach models during post-training before training occurs, achieving R² = 0.9 accuracy in forecasting learned behaviors. The technique uses model interpretability to trace undesired behaviors back to specific data clusters across 260,000 preference pairs in datasets like Dolci and Tulu 3, enabling targeted interventions rather than trial-and-error debugging. This allows practitioners to identify and fix problems like safety regression, hallucinated links, and context-specific sycophancy in a single training run instead of discovering them after deployment.
The Neuron
·
6 days ago
● 4 sources
Goodfire published a collection of research papers on neural geometry and mechanistic interpretability in AI models, covering topics like sparse autoencoders, circuit analysis, and steering mechanisms in language models. The research includes 40+ papers spanning vision models, large language models, and genomic foundation models, with specific applications like detecting rare LLM failures with 30× fewer rollouts and deploying interpretability for PII detection at Rakuten. The work enables practitioners to understand model internals, identify undesired behaviors, and make targeted interventions to improve AI system performance.
TechCrunch AI
·
6 days ago
● 17 sources
Moonshot AI's upcoming Kimi K3 model, expected between 2 trillion and 3 trillion parameters, is projected to match or exceed Anthropic's Opus 4.8 performance according to Financial Times sources. Moonshot is raising fresh capital at a $31.5 billion valuation, up from $20 billion in May, as Chinese open-weight models increasingly close the performance gap with expensive closed-source alternatives from OpenAI and Anthropic. The release is expected in the coming days and reflects growing momentum toward open-source AI models as cost-effective alternatives to proprietary systems.
Ben's Bites
·
6 days ago
● 4 sources
Thinking Machines launched Inkling, its first open-weights model with a 1M-token context window supporting text, images and audio, available on the Tinker fine-tuning platform. The model is positioned for custom fine-tuning as startups increasingly shift workloads from frontier models to self-hosted versions, with alternatives like GLM-5.2 gaining adoption despite lacking vision capabilities. The release reflects a growing market trend toward open-source and customized AI models rather than reliance on leading proprietary systems.
Hugging Face Blog
·
6 days ago
DharmaOCR, a Portuguese-language optical character recognition model, outperformed newer competitors Mistral OCR4 and Unlimited-OCR on a Brazilian Portuguese benchmark through domain-specific training rather than architectural superiority. DharmaOCR scored 0.925 on the Portuguese benchmark while Mistral OCR4 scored 0.798 and Unlimited-OCR scored 0.7587. The specialized model's advantage persists because concentrating all parameters on a single language outperforms distributing them across multiple languages, even as general OCR architectures improve.
TLDR Dev
·
6 days ago
● 3 sources
A company released Inkling, an open-weights Mixture-of-Experts model with 975B total parameters and 41B active parameters, trained on 45 trillion tokens of multimodal data. The model supports a 1M token context window and includes a smaller 12B variant, with both available for fine-tuning on their Tinker platform. Inkling enables developers to customize and deploy models across diverse domains while balancing performance with computational efficiency through controllable thinking effort.
TLDR
·
6 days ago
● 3 sources
Thinking Machines Lab, founded by Mira Murati, released its first AI model called Inkling, a foundation model with 975 billion parameters designed to perform broadly across multiple domains. The model emphasizes cost-efficiency and can be customized through Tinker, a cloud-based fine-tuning tool. The release represents an attempt to compete with established AI giants by offering a more balanced and adaptable alternative.
The Neuron
·
1 week ago
RoboTTT integrates test-time training into robot foundation models to process 8,000 timesteps of visual and motor context, enabling long-horizon manipulation tasks. The model achieves 87% improvement over single-step baselines and completes a five-minute ten-stage assembly task that baseline policies cannot finish. This long-context scaling unlocks one-shot imitation from video, online self-correction, and recovery from physical perturbations during tasks.
The Neuron
·
1 week ago
● 4 sources
Thinking Machines released Inkling, an open-weight multimodal mixture-of-experts model with a 1M-token context window and controllable reasoning effort designed for enterprise customization. The model achieved benchmark performance between Kimi 2.5 and 2.6, positioning it as a competitive alternative to proprietary APIs. This release enables organizations to deploy and customize their own models rather than relying on vendor-specific solutions.
Meta AI Blog
Meta released SAM 3.1, an updated version of its Segment Anything Model that processes video object tracking more efficiently through a technique called object multiplexing. The model doubles processing speed from 16 to 32 frames per second on a single H100 GPU by tracking up to 16 objects in a single forward pass instead of processing each object separately. This enables real-time object tracking in complex videos while reducing GPU resource requirements, making the technology feasible on smaller hardware.
Meta AI Blog
TRIBE v2 is an AI model trained to predict how the human brain responds to visual, auditory, and language stimuli by learning from fMRI scans of over 700 volunteers. The model was trained on more than 700 healthy volunteers presented with diverse media including images, podcasts, videos, and text, and can make predictions for new subjects, languages, and tasks without additional brain imaging data. Researchers can now test hypotheses about brain function computationally, reducing the need for human subjects in experimental studies and potentially accelerating neuroscience discovery.
Meta AI Blog
Brain2Qwerty v2 decodes sentences from non-invasive brain recordings using AI trained on neural signals, without requiring surgical implants. The system achieved 61% word accuracy across nine participants wearing MEG devices while typing, with the best participant reaching 78% accuracy. The researchers released full training code and datasets to enable broader development of non-invasive brain-computer interfaces for people with communication disorders.
Meta AI Blog
● 3 sources
Meta released Muse Spark 1.1, a multimodal reasoning model designed for agentic tasks with improved capabilities in tool use, coding, and computer interaction. The model supports a 1 million token context window and is now available in public preview through the new Meta Model API. Developers can access Muse Spark 1.1 to build agents that handle complex workflows, debugging, and automation across multiple applications without extensive human intervention.
The Batch
● 5 sources
OpenAI released a preview of its GPT-5.6 family of models—including GPT-5.6 Sol, Terra, and Luna—with performance comparable to Claude 5 Mythos, but initial access is restricted to approximately 20 U.S. government-approved organizations. GPT-5.6 Sol achieved 91.9 percent on Terminal-Bench 2.1 for command-line coding and scored 68.3 percent on World-Class Bio tests, a 10-point improvement over the prior generation. The restricted release and safeguards against dangerous biological, chemical, and cybersecurity information mean legitimate developers may face refusals or account reviews when using these cheaper models for security verification work.
The Batch
DeepSeek released an open-weight reasoning model (DeepSeek-R1) that matches OpenAI's o1 performance, triggering a stock market sell-off of Nvidia and other U.S. tech companies. DeepSeek-R1 costs $2.19 per million output tokens compared to o1's $60 per million, a nearly 30-fold price difference. The advancement demonstrates that algorithmic innovation and optimized training can compete with raw computational scaling, shifting focus away from the assumption that more computing power is the only path to AI progress.
Anthropic News
● 3 sources
Anthropic released Claude Sonnet 5, a model designed to handle autonomous tasks like planning and tool use with performance approaching the more expensive Opus 4.8 model. Pricing is set at $2 per million input tokens and $10 per million output tokens through August 31, 2026, then increases to $3 and $15 respectively. The model becomes the default for Free and Pro users and is available across all Claude plans and platforms.
Latent Space
·
1 week ago
● 3 sources
Thinking Machines Lab released Inkling, a 975-billion-parameter open-weights multimodal model with 41 billion active parameters that processes text, images, and audio. The model was pretrained on 45 trillion tokens and supports context windows up to 1 million tokens, with an Apache 2.0 license available immediately on Hugging Face and partner platforms. Inkling ranks as the strongest U.S.-based open-weights model released to date, though independent reviewers note it remains behind top Chinese open models and closed systems on some benchmarks.
Apple ML Research
·
1 week ago
Researchers found that large language models can improve their code generation performance through simple self-distillation, which involves sampling the model's own outputs with specific temperature settings and then fine-tuning on those samples. The method improved Qwen3-30B-Instruct's pass@1 score on LiveCodeBench from 42.4% to 55.3%, with greater improvements on harder problems. The approach generalizes across different model families and sizes without requiring external verifiers, teacher models, or reinforcement learning, offering a new post-training direction for enhancing LLM code generation.
