Measuring Goodhart’s law
OpenAI Blog
OpenAI has begun quantifying Goodhart's law—the principle that metrics lose their validity when used as optimization targets—because the company struggles to measure some of its core objectives directly. The research examines how specific metrics degrade when they become targets for optimization, rather than mere observations. This allows OpenAI to better balance pursuing measurable goals while accounting for the distortion that measurement-based optimization introduces.
Why it matters
Goodhart’s law famously says: “When a measure becomes a target, it ceases to be a good measure.” Although originally from economics, it’s something we have to grapple with at OpenAI when figuring out how to optimize objectives that are difficult or costly to measure.
