arXiv cs.CL
·
18 hours ago
Researchers identified a shared "countdown subcircuit" in Llama-3.1-70B-Instruct that tracks remaining tokens before reaching a target length across diverse tasks like writing sentences of exact word counts, formatting tables, and positioning DNA sequences. The subcircuit uses an identical geometric motif previously observed in other frontier language models on different tasks, indicating the mechanism generalizes across models. This reverse-engineering approach reveals how language models reuse internal computational structures to generalize single behaviors to many different applications.
arXiv cs.AI
·
18 hours ago
Researchers introduced MxGPS, a multiplex graph transformer for power grid modeling that addresses topology overfitting—where models trained on specific grid structures fail on different grids despite strong in-distribution performance. The model uses 1.6 million parameters (12 times fewer than a reference baseline) and achieves zero boundary violation rates across four unseen grid topologies through joint training on state estimation and power flow tasks. This approach enables models to generalize physics-based understanding across different power grid structures rather than memorizing topology-specific patterns.
arXiv cs.AI
·
18 hours ago
Researchers developed a method to preserve semantic structure in Vision-Language-Action models during fine-tuning on robot demonstrations by anchoring action representations to semantic manifolds. The technique achieved up to 18.7% improvement on real-world in-distribution tasks and 21.5% on out-of-distribution generalization. This approach prevents degradation of the rich semantic representations inherited from pretrained vision-language models, improving robot generalization without changing the deployed model.
arXiv cs.AI
·
18 hours ago
Researchers developed a framework to improve text-to-audio models' ability to follow complex instructions by using audio-aware large language models as fine-grained judges to verify that generated audio contains specified events in the correct temporal order. The method uses preference pairs constructed from ALLM feedback to train models via direct preference optimization, and introduces S3Bench, a new benchmark with narrative scenarios for evaluating multi-event temporal instruction following. The approach improved event completeness, temporal ordering, and joint instruction-following accuracy across benchmarks while maintaining audio quality.
arXiv cs.AI
·
18 hours ago
OptCar is a method for adapting generalist vehicle dynamics models to specific platforms while maintaining performance across different terrains, using history-conditioned adaptation and limited real-world data combined with synthetic training. The approach achieves a 55% reduction in trajectory tracking error at 6 meters per second on high-slip terrain compared to a baseline, requiring only 5 minutes of real data per terrain. The adapted model enables model predictive control systems to handle high-speed off-road autonomy with better generalization to unseen conditions than specialist models trained on more extensive single-terrain data.
arXiv cs.AI
·
18 hours ago
Researchers evaluated whether agent-optimization methods maintain their improvements when applied repeatedly to new tasks over time using Terminal-Bench 2.0, finding that most methods degrade when encountering new tasks. RELAI's Verifiable Continual Learning achieved a 76.4% pass rate compared to 66.0% for GEPA and 64.6% for Meta Harness by incorporating regression control into the optimization loop. Only the method with built-in safeguards against overfitting successfully compounded gains across multiple optimization rounds on new tasks.
