Defining and evaluating political bias in LLMs

OpenAI · 1 year ago · 27

OpenAI developed new testing methods to measure and evaluate political bias in ChatGPT by examining how the model responds to real-world political scenarios. The evaluation framework assesses responses across multiple political dimensions rather than a single left-right spectrum to capture bias more comprehensively. This approach aims to identify where the model exhibits skewed outputs and inform adjustments to reduce political slant in future versions.