Self-Compacting Language Model Agents
Apple Machine Learning Research
Self-compacting language model agents were proposed to prevent long agent traces from accumulating stale context that pushes earlier content out of the context window. The method, SELFCOMPACT, reports up to 18.1 point gains on math and uses 30–70% fewer tokens per question. It changes compaction from fixed token-interval summarization to a model-invoked summarization tool plus a lightweight rubric that decides when to summarize or suppress during trajectories.
Why it matters
Long agent traces composed of chains of thought and tool calls accumulate stale content that anchor subsequent generations, and eventually outgrow the context window. Existing scaffolds mitigate it with fixed-interval compaction triggered at a token threshold. Such triggers pay no heed to trajectory structure, risking discard of partial results mid-derivation or mid-search. We propose SELFCOMPACT, a scaffold that allows the model itself to decide when and how to compact. Specifically, it pairs two inference-time elements: (i) a compaction tool the model invokes to summarize the accumulated…
Related stories
Context Engineering Inside the Harness: 4 Mechanisms That Beat Context Overflow and Goal Loss on Long-Horizon Tasks
MarkTechPost · 1 month ago ·
25
Embarrassingly Simple Self-Distillation Improves Code Generation
Apple · 2 months ago ·
55