Interacting with LLMs with Minimal Chat

Eugene Yan has a nagging suspicion that the chat box everyone slapped onto their LLM demo last year is the wrong default. His argument is simple: people already navigate the web by clicking images, filtering results, and letting apps infer things from behavior. Google Maps knows where you are. Netflix knows what you've watched. Amazon knows what you bought last Tuesday. None of that requires typing a sentence. So why, he asks, should talking to an LLM suddenly demand you type out everything it needs to know?

To test the idea, Yan built a small book-recommendation prototype that leans hard on clicking and light on chatting. You pick a book you like, and the system pulls similar titles using nearest-neighbor search over item embeddings trained on a product graph built from e-commerce data, with random walks and representation learning doing the heavy lifting. Recently viewed books get extra weight if you've browsed a few in one session. From there you can narrow things down with pre-cached vibe keywords, tags pulled from book descriptions and sources like the UCSD Book Graph, rather than anything an LLM generates on the fly, since that's still too slow to do live.

Only at the end does an actual language model show up, acting as a librarian who already knows your session history. Ask it for "more books by female authors" and it doesn't need a paragraph of context from you. It already has it, because your clicks did the explaining. Yan wired the whole thing together with FastAPI and Jinja templates on the backend, and used streaming responses over async Python and aiohttp so the chat reply doesn't feel like it's crawling out of a modem.

What's notable here isn't the tech stack, which is fairly standard recsys plumbing. It's the sequencing: context first, chat second, treating the LLM as the last mile rather than the whole interface. Yan floated this on Twitter back in April 2023, wondering aloud whether shoppers browsing electronics or clothes really want to describe what they want in prose when they'd rather just look at specs and photos. A year-plus later, with agents and voice assistants multiplying, that question hasn't gone away. If anything it's gotten louder, as more products realize that typing a paragraph to get a one-line answer is often just friction wearing a friendly face.