The distillation panic
Interconnects Nathan Lambert
Chinese labs have used API jailbreaking to extract model outputs for training, prompting U.S. policymakers to target the technique of "distillation," which is a standard industry practice for creating smaller models from larger ones. Congress has moved bills through committee and executive orders are pushing for action, with potential regulatory consequences including restrictions on open-weight models trained on Chinese systems. Overly broad regulation risks permanently damaging the Western academic and open-source AI ecosystem while doing little to stop determined competitors, since the core problem is API abuse rather than distillation itself.
Why it matters
‘Distillation attacks’ is a horrible term for what is happening right now.
