Towards developing future-ready skills with generative AI

Google Research just put a number on something educators have argued about for decades: how good are you, really, at resolving conflict or building on someone else's idea? The new tool, called Vantage, drops students into simulated conversations with AI avatars, then scores them on skills the OECD and the World Economic Forum keep flagging as essential for the next few decades, things like critical thinking and creative collaboration. It's live now in English through Google Labs, aimed at high school and college students.

The mechanics are more clever than they first sound. An "Executive LLM" runs the conversation and quietly steers it, according to Google, deliberately introducing friction, a pushback here, a disagreement there, so the student actually has to demonstrate the skill being measured rather than coast through a pleasant chat. Once the task wraps, a separate AI Evaluator reads the transcript against a rubric and produces a skill map: a score plus written feedback. It's essentially a testing engine built for things that don't fit neatly into a testing engine.

Google didn't just ship this and hope for the best. Working with New York University, the team ran a study with 188 people aged 18 to 25, checking two things: whether the steering actually worked, and whether the AI Evaluator's scores lined up with human raters. On the first question, steered conversations produced noticeably more usable evidence of the target skill than unsteered ones, without feeling scripted. On the second, the AI Evaluator's agreement with NYU's human graders matched the agreement between two human graders with each other. A second study with startup OpenMic, covering 180 students on creative writing tasks like character interviews, found similarly strong correlation between AI and human scoring.

The bigger ambition here is a