Standard Intelligence: Training General Intelligence in Pixel Space
Sequoia amoore ● Covered by 2 sources
A startup called Standard Intelligence just raised a Series A to train AI agents on raw video of computer screens instead of text or code. It's a bet that watching pixels, not reading language, is the real path to agents that can actually use a computer.
Based on reporting by Sequoia, amoore — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Most of the AI agent world right now is obsessed with language models wrapped in tool calls, screenshots, and increasingly baroque harnesses. Standard Intelligence, a six-person startup out of San Francisco, is going the other direction entirely. Their model, FDM-1, doesn't read instructions or parse screenshots through a vision adapter bolted onto an LLM. It watches raw video of screens and predicts the next mouse movement, click, or keystroke, the same way Tesla's Full Self-Driving stack learns to steer from camera footage rather than hand-coded rules.
The scale of what they've built to support this is the real story. The team assembled an 11-million-hour dataset of computer-use footage, which they say is the largest of its kind anywhere. To make training on that much video tractable, they built a video encoder roughly 50 times more token-efficient than existing approaches, letting nearly two hours of 30-frames-per-second footage fit into a single 1-million-token context window. They also built their own 30-petabyte storage cluster in San Francisco for under $500,000, about a twentieth of what a hyperscaler would charge for the equivalent.
What FDM-1 can already do is genuinely odd for a model trained this way. It extrudes a CAD gear inside Blender. It drives a virtual car around a San Francisco block after just an hour of fine-tuning. It hunts for software bugs by exploring a program's states with something like human curiosity, rather than following a scripted test suite. None of that came from prompting tricks or elaborate agent scaffolding — it came from predicting pixels at scale and letting generality show up on its own, which is the same
My take — AI-written commentary, not fact-checked reporting
Betting a whole company on raw video pre-training when most of the field has converged on LLM-plus-tools is either genuinely visionary or a very expensive way to relearn why everyone else went the other route — probably a bit of both. I like that they built their own cheap storage and a 50x encoder instead of just throwing more GPUs at the problem, that's the kind of scrappy engineering the industry has gotten lazy about. But turning a Blender demo into something that reliably automates knowledge work is a much longer road than this writeup lets on.
Read more about this at: Sequoia
