Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models
arXiv cs.CL 18 hours ago
Researchers developed function-aware fill-in-the-middle mid-training, a technique that masks functions in code based on program dependency analysis to help coding agents better integrate external tool outputs. Mid-training Qwen models on 2.6 billion tokens from 968 GitHub repositories improved performance on SWE-Bench-Verified by +2.8 to +3.2 points depending on model size, with larger gains of +3.7 to +5.4 on SWE-Bench-Lite. The approach preserves general coding ability while improving agent performance, as the function-call inductive bias transfers across different benchmarks and post-training methods.