LLM-powered Biographies
Eugene Yan
A data scientist asked six different AI models to write his own biography, just to see what happened. The results ranged from mostly-right to wildly fictional, showing how much these models just guess.
Based on reporting by Eugene Yan — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Eugene Yan decided to run a little experiment on himself. Curious about what large language models actually know versus what they confidently invent, he asked several of them, from OpenAI's GPT-4 down to Cohere's command-xlarge, to write his biography. He's a real person with a real, googleable career, so this doubled as a decent litmus test for how these systems handle facts about someone who isn't Einstein or Elon Musk.
GPT-4 came closest to reality, nailing his first two jobs but inventing degrees from the London School of Economics and NYU Stern that don't exist, plus a fictional stint at Amazon. GPT-3.5-turbo did worse, giving him a Stanford master's, a Google job he never had, and a startup called Axon he never founded. Text-davinci-003 abandoned reality entirely, turning him into a Toronto-born venture capitalist who backed Uber and Dropbox. Older davinci from 2021 didn't even attempt a biography, instead spitting out a broken, repetitive list of
My take — AI-written commentary, not fact-checked reporting
favorite loves
Read more about this at: Eugene Yan