The next hurdle for AI agents: getting websites to let them in
TechCrunch Sarah Perez
AI agents are trying to buy things for people, but websites keep shutting them out. That’s pushing companies toward new rules for bot-to-business access.
Based on reporting by TechCrunch, Sarah Perez — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Personal AI agents are moving beyond chat. Meta’s Muse, Instinct, ChatGPT’s Dots and similar tools are being pitched as helpers that can book flights, order groceries and reserve dinner without the user doing the legwork themselves. That promise hits a wall fast when the website on the other side decides the agent doesn’t get in.
Amazon recently blocked Muse from its retail site, cutting off browsing and purchases from its product catalog. But Amazon is only the loudest example. Across social media, users keep running into the same sort of failure, sometimes because a site has deliberately shut the door and sometimes because its anti-bot systems can’t tell a helpful agent from spam.
Walmart is a good case for the confusion. TechCrunch saw complaints saying Muse couldn’t finish purchases there, and NBC reported similar trouble in tests. Walmart says the block wasn’t intentional. It is, in fact, partnered with Muse after Meta announced the tie-up at Connect in September. The problem, Walmart says, is that its human-verification step can break if the flow is interrupted, which leaves the agent booted out.
That friction is now pushing companies toward standards. Meta, Walmart, Stripe, Sierra, Genesys, Rocket, NiCE and Decagon have started work on an open protocol for how AI agents should talk to businesses in online commerce. Meta says the goal is to separate the useful bots acting for customers from the malicious ones. That distinction matters, because the current web-security playbook was built for a different internet.
Not every company wants in. Delta says it is guarding against unauthorized automated activity and would only open up to agents with security and customer experience in mind. United points to terms that ban robots, spiders and other automated devices without written permission. Yelp says non-human traffic isn’t allowed unless the agent pays for access through its data licensing program, which means a bot trying to get a quote, join a waitlist or book a table is likely to fail without a formal deal.
Cloudflare sits in the middle of the fight, and not exactly as a neutral referee. Its tools can block AI agents, it changed crawler defaults on September 15, and some people suspect that shift is affecting Muse traffic. The company says it has no specific data to share on these complaints, but the bigger picture is hard to miss: the web is starting to sort bots into the good kind and the unwelcome kind, and it does not have a clean system for doing that yet.
My take — AI-written commentary, not fact-checked reporting
The obvious fix here is not more bot theater and not a pile of broken human-verification buttons. It’s clearer permissions, real business agreements and fewer websites pretending every automated request is either a thief or a saint. The web spent years teaching machines to behave badly, so now it has to learn the difference the hard way.
Read more about this at: TechCrunch