Patreon stops asking AI bots not to scrape — and starts blocking them

Patreon spent the last couple years telling AI crawlers to stay away from creator content using robots.txt, the internet's version of a polite sign asking people not to walk on the grass. Thursday's announcement admits what everyone suspected: a lot of bots just walked right over it anyway. Now the company is teaming up with Cloudflare to actually enforce the boundary instead of just posting it.

The timing tracks with two things happening at once. AI scraping tools have gotten more sophisticated since 2023, better at slipping past the kind of soft blocks Patreon originally relied on. And Patreon itself has been adding features — a redesigned Home Feed, short-form posts called Quips — that surface more content outside the paywall, which is great for discovery and also great for anything trying to hoover up training data.

The mechanism here is Cloudflare's AI Crawl Control, part of a broader push by the infrastructure company to give publishers real leverage against AI firms. Cloudflare already runs Pay Per Crawl, letting sites charge bots for access, and this month it started blocking so-called mixed-use crawlers by default on any page carrying ads — the ones that both index content for search and quietly feed it into training pipelines. Patreon is leaning on that same infrastructure but drawing a harder line: indexing bots that send traffic back to creators are still welcome, training bots are not, full stop.

What's notable is Patreon's own framing. Product chief Drew Rowny didn't just say the company wants to protect content — he said creators shouldn't have to accept AI training as the toll for building an audience, which is the deal almost everywhere else on the internet right now. During testing, individual crawlers' access attempts fell from thousands per week to zero, a number that says less about Patreon's cleverness and more about how brazenly those bots had been ignoring the rules until someone finally closed the door.