Tracing distinctive language in AI-written text
Allen Institute (AI2)
Researchers used a free tool to check if AI-written books recycle phrases from real, published books. Turns out AI-heavy bestsellers use way more rare book phrases than clean ones do.
Based on reporting by Allen Institute (AI2) — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
It started with a mystery over a prize-winning short story. Readers of a Commonwealth Foundation contest suspected the winning entry had been written by AI, an episode that got nicknamed GrantaGate. That suspicion pushed Tuhin Chakrabarty, a computer science professor at Stony Brook University, toward a more interesting question than "is this AI or not." He wanted to know where the language itself came from.
An AI detector can only spit out a probability. It can't point to a source. So Chakrabarty turned to Ai2's infini-gram, a search engine that indexes enormous text datasets and can tell you exactly how often any phrase shows up across them. When his student Xinyue Liu ran the GrantaGate story through it, a phrase describing a "sour tang of fermenting" turned out to echo language from a fan-fiction site. Not proof of copying, exactly, but the kind of circumstantial evidence a bare detector score can't offer.
That led to a bigger study. Chakrabarty's team cross-referenced Google Books with infini-gram to isolate genuinely rare expressions — phrases showing up in five or fewer Google Books volumes and nowhere in infini-gram's web snapshot. That filters out common stock lines and leaves language tied specifically to published books. They then applied this to top-selling self-published genre books on Amazon, split into groups with and without substantial detected AI text.
The numbers are the real story here. Among the 100 highest-revenue books in each group, rare book expressions covered 43.2% of text flagged as substantially AI-written, versus 37.6% for the non-AI group. Widen the pool to 200 books and the gap narrows but stays statistically significant. Compare either group to award-winning or nominated literature, though, and the contrast sharpens dramatically — that literary set averaged just 19.1% rare-expression coverage, far below both Amazon groups, and especially below the 41.6% seen in the AI-heavy books.
Chakrabarty argues this doesn't prove plagiarism in any legal sense, but it does undercut a common defense — that AI training on books is just like a person reading books. Rare phrases showing up disproportionately in AI-flagged commercial writing looks less like inspiration and more like leakage from training data. He's now extending the method to copyright research, trying to trace whether reproduced book snippets trace back to pirated copies or to manuscripts that leaked online before publication. None of this works without an open, searchable index of what these models may have actually trained on — which is exactly the gap infini-gram and Ai2's open datasets like Dolma are built to fill.
My take — AI-written commentary, not fact-checked reporting
This is what accountability in AI actually looks like — not vibes-based detection scores, but someone building the receipts. The fact that AI-heavy bestsellers lean so much harder on rare, book-specific phrasing than award-winning human writing does should worry anyone who thinks synthetic content is a victimless shortcut. Open, searchable training data is the only reason this kind of forensic work is even possible, and that's a strong argument against locking training corpora behind closed doors.
Read more about this at: Allen Institute (AI2)