Plan, divide, and conquer: How weak models excel at long context tasks
Together AI
Researchers developed a Divide & Conquer framework where smaller models split long documents into chunks, process them in parallel, and aggregate results, showing that Llama-3-70B and Qwen-72B can match or exceed GPT-4o single-shot performance on tasks like QA and summarization. Testing on diverse long-context tasks found that optimal chunk size can be identified with just 5 random samples, reducing computational cost and latency compared to processing massive context windows serially. The approach works for moderate cross-chunk dependency tasks but fails when subtle context connections span the entire document, limiting applicability to specific use cases.
Why it matters
As context windows grow, LLM performance degrades in unexpected ways. We show how a "Divide & Conquer" framework — breaking long documents into parallel chunks with a planner, workers, and manager — lets smaller models like Llama-3-70B and Qwen-72B outperform GPT-4o single-shot.
