Plan, divide, and conquer: How weak models excel at long context tasks

Together AI · 3 months ago

Researchers developed a Divide & Conquer framework where smaller models split long documents into chunks, process them in parallel, and aggregate results, showing that Llama-3-70B and Qwen-72B can match or exceed GPT-4o single-shot performance on tasks like QA and summarization. Testing on diverse long-context tasks found that optimal chunk size can be identified with just 5 random samples, reducing computational cost and latency compared to processing massive context windows serially. The approach works for moderate cross-chunk dependency tasks but fails when subtle context connections span the entire document, limiting applicability to specific use cases.