Make AI Run a Gauntlet Against Real-World Work

Somewhere around 55,000 lines of code, split across eleven subsystems, a fleet of AI agents working under a shared orchestration contract put together a first-person shooter that runs in a browser tab on WebGL2 and Three.js r180. There isn't a single image file, 3D model, or audio clip anywhere in it. Every wall texture, every gunshot, every character skin gets generated procedurally at load time — 19 surface types forged on the GPU, atmospheric sky and fog computed live, weapons and ragdolls simulated with physics written from scratch rather than pulled from a library.

The game itself, a market-street map with a few hundred props and skinned soldiers running navmesh AI, is impressive as a party trick. But the more interesting part of this project isn't the shooter — it's the harness the team built to keep the agents honest. Tools like baseline.mjs and imagediff.mjs exist to catch a specific failure mode: agents claiming progress that a per-pixel diff proves didn't happen. And it caught real problems. A static-camera benchmark reported a smooth 94 fps while actual gameplay, measured at full Retina resolution with AI and gunfire active, was stuttering along at 12 to 17 fps with stalls over a full second, caused by more than 30 shader programs compiling mid-frame. Median frame time had been hiding the whole mess.

Fixing that meant building a shader pre-warm step, and the team held itself to a strict rule while doing it: the optimized build had to be pixel-for-pixel identical to the pre-optimization one, verified across all eleven reference shots rather than eyeballed. That discipline paid off — worst-case frame times dropped from over a second to under 100 milliseconds, and boot time roughly halved — but it also exposed a second issue. Their own screenshot tool had been reusing one browser page across shots, letting particle effects and exposure settings leak between captures, so two supposedly identical runs disagreed on 10 of 11 images. Isolating each shot in a fresh page fixed that and made the whole comparison trustworthy.

When the team finally pointed adversarial AI critics at the results and asked them to judge the game against a modern Call of Duty, the answer was blunt. Scores crept from 3.59 up to 5.05 out of 10 across rounds, two shots got labeled 'CLOSE,' the rest stayed 'AMATEUR,' and in blind side-by-side comparisons every single critic, every single round, correctly picked out the real Call of Duty frame. The gaps are specific and unglamorous: hands that look like blocky slabs gripping a gun, surfaces that read as generated noise up close, distant enemies that look like mannequins, and a lighting bug where the weapon-view rig throws roughly 20 times the light per unit of surface color that the rest of the world gets, forcing the team to fake weapon colors just to compensate.

The most telling result, though, came from process rather than pixels. Splitting work across six agents, each owning one directory in parallel, nudged the score up slightly but actually made things worse under the hood, because lighting, tone-mapping, and sky are one interconnected system that isolated agents kept quietly breaking for each other. Switching to a single sequential pass, one owner per coupled problem, more than doubled that improvement and cut major defects by more than half. And the single best fix in the whole project came from an agent that ignored its brief: critics had spent three rounds insisting the weapon looked 'untextured,' when the real issue was a lighting imbalance crushing the diffuse color. Following the brief would have made it worse.