AI Engineer 2024 Keynote - What We Learned from a Year of LLMs

Six speakers, one stage, one talk. That's the setup Eugene Yan and his co-authors on "What We've Learned From A Year of Building with LLMs" walked into for the closing keynote at AI Engineer World's Fair 2024. Wrangling that many voices into something coherent is its own engineering problem, and Yan treats it like one.

The team actually drafted two competing formats as formal one-pagers before picking a winner: a traditional panel versus what they called 2x2x2, pairs of speakers taking the stage in sequence. They went with the pairs. Yan says it took more prep and more coordination, but it paid off in a tighter, higher-energy session than a six-person panel discussion would have produced. He's not rushing to repeat the experiment with a crowd that size anytime soon, though he came away with real lessons about multi-speaker talks.

The bigger curveball came separately. Swyx, the conference organizer and a close collaborator of Yan's, couldn't make it in person, which left a gap in the schedule for a fireside chat titled "Copilots Everywhere." Yan stepped in to host it opposite GitHub CEO Thomas Dohmke. Lucky timing helped: a VC had arranged dinner between the two just the week before, so there was already some rapport. Still, Yan admits interviewing someone at that level rattled him, and he leaned hard on last-minute coaching from Swyx, Ben, and Hamel to get through it.

What comes through in his recap isn't a triumphant conference wrap-up so much as an honest account of logistics under pressure. Finalizing keynote slides the night before, running on fumes from back-to-back conversations and social events, then still delivering both a six-person keynote and an unplanned CEO interview. Yan calls the whole trip a highlight and says he's already planning to be back for the next one.