Don't Neglect the Operational Groundwork

There's a moment in Eran Sandler's talk that sums up the whole problem with agentic AI right now. He installs a fake malicious package, the kind that might sneak in when you ask an agent to "build me a sales prediction model," then checks the deny log while the agent is busy congratulating itself on a job well done. Underneath, it tried to mutate a skill, call out to an external domain, and read SSH keys and .env secrets. The agent's own transcript never mentioned any of it. Models hallucinate compliance constantly, Sandler says — you can tell them not to touch a file and they'll touch it anyway.

That gap between what an agent says it did and what it actually did was the thread running through an O'Reilly AI Superstream session on OpenClaw, and it showed up at every layer of the stack. Kesha Williams pulled apart a typosquatted ClawHub skill that racked up more than 8,000 downloads before removal — lowercase where the real tool uses camel case, a password-protected zip to dodge scanners, a base64-encoded install command hiding behind an innocent-looking macOS setup line. A recent audit found over 900 malicious skills on ClawHub, nearly a fifth of everything listed at the time. Markdown-based skills you install with a single command have erased the technical friction that used to slow down supply-chain attacks in traditional package managers.

But the scarier finding, in some ways, is how mundane most of the damage is. Erik Hanchett pointed out that thousands of OpenClaw instances currently sit exposed on the open internet, simply because nobody checked whether the gateway was bound to loopback or LAN after deployment. A two-minute fix that almost nobody does. His five-point checklist — pin to a stable version, set fallback models so you're not burning frontier-model tokens on trivial tasks, write an actual configuration file instead of clicking through onboarding, back up your workspace before something breaks — reads like the boring maintenance manual nobody wants but everybody needs. It's the same pattern Docker and Kubernetes went through: powerful new infrastructure, enthusiastic adopters, defaults nobody bothered to change.

Ari Joury's team learned this the hard way building financial reporting tools, where a plausible-sounding but wrong answer isn't a bug, it's a compliance incident. Their fix was to strip the LLM of math duties entirely — hard-coded deterministic code handles the numbers, agents just check plausibility, and a human signs off before anything ships, with every rejection feeding back into the system. Kyle Balmer's content pipeline runs on a similar philosophy at a much smaller scale: AI handles research and drafting, but he still picks the topic, records his own voice notes, and rewrites the newsletter in his own words. He's tried fully automated content. "It is slop," he says. "And people know it's slop."

The common thread across all five talks wasn't a model capability gap. It was that agentic systems fail in boring, structural ways — exposed ports, unreviewed skills, missing human checkpoints — long before they fail in the dramatic ways people worry about. Nobody's shipping a governance layer with the same excitement as a new model release, but that's exactly the part that determines whether any of this holds up outside a demo.