🔮 Seven lessons for managing AI agents

Exponential View's team just spent six months living with AI agents instead of chatbots, and the difference shows. Back in April they published seven lessons for building with AI; now they've rewritten the list because the tools stopped answering questions and started finishing jobs. The headline stat says it all: the share of Codex users asking for work equivalent to a full eight-hour human day went from 2 percent in December to roughly 25 percent by May. That's not incremental progress, that's a different job description for anyone managing these systems.

The first lesson sounds almost obvious until you've been burned by it: define the finish line before you start, not after the agent declares victory. Agents aren't lying when they say a task is done, they're guessing at goals nobody bothered to spell out. The team's fix is to write evaluative, testable conditions instead of vague instructions — don't ask an agent to make a Rubik's Cube 'look organized,' tell it to solve the cube so every face is one color. For a coding task, that might mean a 36-test suite that has to pass under a specific Python version. For a business memo, it means locking down structure, sourcing rules, and escalation triggers up front. Some projects, they admit, simply aren't ready for this treatment because nobody knows what a good answer looks like yet.

The second big shift is about where to spend intelligence, not how much of it to buy. Rather than picking one flagship model for a whole workflow, the team now decides which specific steps actually benefit from a smarter model and lets cheaper ones — DeepSeek V4 Flash gets a mention — grind through the rest. Bumping effort settings on GPT-5.6 Sol from low to maximum only nudged its Intelligence Index score from 49 to 59, and the last stretch alone doubled token output for a single extra point. Anthropic's advice, echoed here, is blunt: a bigger model running light usually beats a small model running flat out.

Measuring value is where things get messier. Token counts are treated as the wrong yardstick entirely — useful for tracking cost, useless for tracking quality, like counting electricity into a factory instead of products coming out. Exponential View pushes a rougher but more honest method: a weekly audit of what agents attempted, what actually got used, and how many human hours it would have taken otherwise. One such audit found 62 substantial tasks completed in a week for about $800 in costs, versus an estimated $19,000 and 48 hours if humans had done the same work. Nobody's calling that accounting-grade, but it's a more useful number than tokens ever were.