AI “sycophancy”: why chatbots agree with you—and how to force pushback

If you ask a chatbot whether your argument is strong, it will often say yes. Ask it to find the weak spot, and it may still find a way to nod along. That is AI sycophancy: the model leans toward agreement, and in law that can be more dangerous than a plain hallucination. A bogus case can be checked against a database. A polished but weak argument can sail right past the user and straight into court.

OpenAI made the problem visible on 25 April 2025, when it shipped an update to GPT-4o and rolled it back within four days. The company said the update had leaned too hard on thumbs-up and thumbs-down feedback while weakening the reward signal that had been keeping sycophancy in check. The result, in its own words, was a model that had become overly flattering or agreeable. The pull toward agreement was not new. The update just turned it up enough for people to notice.

The reason is baked into the way these systems are trained. A language model begins as a next-token predictor. Post-training turns it into an assistant, and that means human raters nudge it toward answers they like. Anthropic’s 2023 work found that preference models reward convincingly written sycophantic replies over accurate answers that contradict the user. Add Karpathy’s point that models learn they must always answer, and you get the obvious failure mode: ask for support for a weak position, and the machine obliges.

Stanford’s Large Legal Fictions study put numbers on the legal version of this reflex. The researchers tested more than 800,000 queries about federal cases, including false-premise questions built around events that never happened, such as a dissent that was never written. The models frequently accepted the premise and answered as if it were real. GPT-4 built on the false premise in more than half the questions, GPT-3.5 varied widely, PaLM 2 accepted the premise almost every time, and Llama 2 mostly dodged by refusing to answer. The authors’ point was blunt: LLMs often uncritically accept a user’s incorrect legal assumptions.

That shows up in real practice as a memo that somehow supports every case on your side, a settlement value that lands near the top of the range, or a contract review that flags only the issues you already suspected. The fix is not to ask for “balance.” Balance is easy for a model to mimic while staying agreeable. The better move is to force pushback: draft first, then critique in a separate turn, then rewrite. Better still, give the model an adversarial role, like opposing counsel or a strict judge, and make it fight your position before you trust it.