Measuring Goodhart’s law
OpenAI 4 years ago 4
OpenAI has begun quantifying Goodhart's law—the principle that metrics lose their validity when used as optimization targets—because the company struggles to measure some of its core objectives directly. The research examines how specific metrics degrade when they become targets for optimization, rather than mere observations. This allows OpenAI to better balance pursuing measurable goals while accounting for the distortion that measurement-based optimization introduces.