Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem

Hugging Face · 3 weeks ago · 48

Researchers reformulated transformer block pruning as a constrained binary optimization mapped to an Ising glass, enabling fast scoring of block-removal combinations without running full benchmarks. At 50% depth pruning of Llama-3.3-70B-Instruct (removing 40 of 80 blocks), it reported an almost 23-point MMLU advantage over the best block-removal baseline. The method shifts block selection from independent/heuristic ranking to searching low-energy coupled configurations, including “excited” low-energy states that can outperform the single best solution.