Is AI making your teams better, or just busier?

There's a familiar pattern in corporate AI rollouts: everyone gets access, usage numbers climb, someone builds a Slack channel to celebrate wins, and six months later the team is doing the exact same work, just marginally faster. Ably's engineering leadership decided this pattern is a measurement failure, not an adoption failure, and they've built a system to prove it.

The numbers back up their skepticism. McKinsey's 2025 State of AI survey found that 88% of companies use AI in at least one business function, yet only 39% report any bottom-line impact — and even those gains are typically under 5% of EBIT. That gap between widespread usage and negligible results is the whole problem. Usage tells you someone opened a tool. It says nothing about whether the work itself got better.

Ably's answer is two KPIs, scored 1 to 5 monthly by team leads, rolling up from individual to team to company level. The first tracks whether AI is unlocking outcomes that weren't achievable before. The second tracks how embedded AI actually is in daily delivery — automated workflows, output that couldn't have been produced solo, at that speed or quality. Crucially, scores require evidence. A team lead who can't find a concrete example doesn't get to bump the number, because the goal is documenting real change, not rewarding enthusiasm.

What's notable is how unglamorous the honest starting point looks. Ably's first scoring round in January landed at 2 across both KPIs, with a few teams below that. Leadership treated this as useful information rather than a failure — a real baseline instead of an inflated one. They've since pushed most teams to 3, with 4 and 5 as year-end targets, and they're backing the scoring with structural changes: AI expectations built into promotion criteria, a shared internal skill repository accessible through their MCP setup, and something they call an