AI Inference Optimization

2 summarised stories about AI Inference Optimization, each linking back to the original source. Browse all topics →

Thursday, 16 July 2026

Amplitude-Only FFN Intervention for Tool-Structured LLM Inference Method: Gated Evaluation Protocol, and Cross-Model Empirical Results

arXiv cs.CL 18 hours ago

Researchers propose Amplitude Gating, an inference-time technique that modulates feed-forward network activations in large language models to improve structured outputs like tool calls without retraining. On Qwen3.5-9B, the method improved tool-structured task performance from 38.66% to 42.92%, and on Qwen3-8B achieved an 11.36 percentage point improvement on JSON formatting tasks. The approach enables safer optimization of model inference for tool-using agents while preserving pretrained weights, though effectiveness varies by model and requires category-specific configuration.

AI Agents AI Code Generation AI Inference Optimization Qwen

Attention-Discounted Adaptive Sampler for Masked Diffusion Language Models

arXiv cs.CL 18 hours ago

Researchers proposed ADAS, a training-free reranking method for masked diffusion language models that improves token selection by accounting for attention patterns between candidate positions. ADAS achieved 9.11 and 10.46 percentage point improvements on GSM8K, MATH500, HumanEval, and MBPP benchmarks when integrated with existing samplers at matched evaluation costs. The method enables faster parallel decoding with better prediction accuracy by using attention as a soft penalty rather than hard constraints.

AI Algorithms AI Inference Optimization Diffusion Models Language Models