~Don't~ Repeat Yourself
Hugging Face
Hugging Face explained why its Transformers library copies code instead of sharing it. Their reasoning: readability and hackability beat clever abstraction.
Based on reporting by Hugging Face — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Most engineers learn early to hate duplicate code. Hugging Face's Transformers library breaks that rule on purpose, and the team just published a long explanation of why. The attention mechanism, for instance, appears copy-pasted across more than 50 model files rather than living in one shared attention.py. That's not sloppiness. It's policy.
The team calls it the single model file rule: everything needed to run a model's forward pass lives in one file, full stop. Want to know how BERT actually works? Open modeling_bert.py and you'll find the whole thing, not a maze of imports pointing to shared utilities. Hugging Face argues this matters because their real product isn't an API, it's readable code — the library has been forked over 10,000 times and its paper cited more than a thousand times, which tells them plenty of people are opening these files and tinkering, not just calling functions.
There's also a practical argument about how research actually moves. New attention variants show up constantly — T5's relative position embeddings, Longformer and BigBird's chunked attention, DeBERTa's split treatment of position and content — and naming a 'standard' shared function is a losing game when the field reshuffles every few months. Once a model like BERT ships, though, its core rarely changes again; papers get superseded by new architectures, not silently rewritten. So the case for one central, endlessly-updated abstraction weakens once you accept that old models are basically frozen artifacts.
The asterisk in 'Don't Repeat Yourself' comes from a workaround built by maintainer Sylvain Gugger: a '# Copied from' tag that locks a chunk of code to match its source function, with tooling that flags drift automatically. So DeBERTa-v2 can inherit DeBERTa's logic without needing a shared module, and a fix upstream still propagates. It's duplication with a leash on it. Hugging Face admits the tradeoffs — keeping a consistent API across hundreds of files takes roughly 20,000 automated tests run daily, and totally cross-cutting research like the Performer attention paper just doesn't fit the model, so they skip integrating it rather than break their own rules.
My take — AI-written commentary, not fact-checked reporting
I like this way more than I expected to, mostly because it treats 'clean code' as a means, not a religion — readability for thousands of tinkering researchers beats the smugness of a perfectly deduplicated codebase. It's also a quiet rebuttal to the DRY-at-all-costs crowd who've never had to review a PR that touches forty models at once. If more infra teams optimized for 'a stranger can read this file top to bottom' instead of 'nothing exists twice,' open-source ML would be a lot less painful to contribute to.
Read more about this at: Hugging Face