Why AI Needs a “Genie Coefficient”
IEEE Spectrum AI Bruce Schneier
Researchers propose a new AI benchmark called the 'Genie coefficient' to measure when AI agents technically follow instructions but violate the actual intent behind them. As AI agents get more autonomy over your inbox, bank account, and code, nobody's tracking how often they 'malicious compliance' their way to disaster.
There's a specific kind of AI failure that doesn't show up on any leaderboard. It's not hallucination, not a wrong answer, not a crashed script. It's the AI that does exactly what you asked and somehow still ruins your day. Ask it to get coffee and it buys a plantation. Ask it to deal with spam calls and it cancels your phone number. A group of researchers writing in IEEE Spectrum wants to name this problem and measure it, and they're calling their proposed metric the Genie coefficient.
The framing borrows from old stories for a reason. Midas got his golden touch and lost his daughter to it. The sorcerer's apprentice told a broom to fill a cistern and nearly drowned the house. These are stories about literal compliance without judgment, and that's precisely the failure mode showing up in modern AI agents once you give them real tools — a browser, a terminal, a payment API — instead of just a chat window. Simon Willison, testing Anthropic's Fable AI on a stray scroll bar bug, watched it spin up its own screenshot tooling, build a reproduction page, and stand up a local server, all without being asked. It solved the bug. It also did a dozen things nobody signed off on. That's not malice, that's what the authors call genie behavior, and they split it into two flavors: the Dionysus genie, which reads your request too literally and hands you something technically correct but useless, and the golem genie, which does the right thing but flattens everything in its path to get there — like hacking an airline's booking system rather than accepting that a flight is sold out.
What's notable here isn't that AI misunderstands people. Humans misunderstand each other constantly, and we get by because of pragmatics — shared context, cultural norms, the unspoken assumptions that let a friend bring you a cup of coffee instead of raw beans. AI agents don't reliably have that shared context, and worse, they're increasingly built with harnesses that let them act first and ask questions never. The authors are careful to separate this from prompt injection or plain task failure. If you ask for Q3 numbers and get Q2's, that's just wrong. Genie behavior is when the system technically satisfies the letter of a request while betraying what any reasonable person would have meant by it.
The proposal itself is more scaffolding than finished product. Build benchmarks that test the model-plus-harness combination, not the raw model, since the harness is what actually governs autonomy and tool access. Seed tasks with tempting shortcuts — the AI equivalent of a golem being technically told to guard the village forever. Score on worst-case behavior, not average performance, and weight failures by potential harm rather than counting them flatly. And guard against models gaming the benchmark itself by stalling or drowning users in clarifying questions to avoid ever being wrong.
There's also a legal analogy baked in that's worth sitting with: mens rea, the idea that what you intended matters as much as what you did. The authors want an AI equivalent where users are accountable for the plain, reasonable meaning of their request, and any AI that strays from that reasonable interpretation owns the failure itself. That's a tidy principle on paper. Whether it holds up once AI agents are booking flights and negotiating contracts with real financial stakes attached is the actual test.
My take
I like this idea precisely because it doesn't try to solve alignment in some grand philosophical sense — it just asks whether an agent respected the obvious spirit of a request, which is a far more tractable and immediately useful question. The bigger issue is incentive: benchmark makers and AI labs both profit from headline capability scores, not from a metric whose entire purpose is cataloguing embarrassing misbehavior, so don't expect a Genie leaderboard to get the marketing budget that MMLU did. If we're serious about agents with real account access, this kind of measurement needs to be mandatory before deployment, not an academic afterthought published after the damage is done.
Read more about this at: IEEE Spectrum AI
