Launch HN: Agnost AI (YC S26) – Extract user feedback from agent conversations

Shubham and Parth spent a year building around AI agents before they landed on the actual problem worth solving: nobody building a chatbot or voice assistant has any real idea whether their users are happy. Web apps have decades of analytics tooling built around clicks and funnels. Conversational products have none of that, because the entire interface is language, and language is messy. Users don't file bug reports. They curse at the bot, ask the same question five different ways, correct it mid-sentence, or just quietly leave. Agnost AI is built to catch exactly those moments.

The product ingests raw conversation transcripts via SDK or OTel, then clusters them into intents specific to each customer's product. Bugs and feature requests come as default buckets, but most categories get created dynamically as patterns emerge in the data, and users can spin up their own clusters just by describing them in plain English. One customer, an AI video editing startup, used it to discover that roughly 70 users were asking for auto-subtitles, but the request never showed up cleanly because people phrased it a dozen different ways: 'add this text in this frame,' 'can you caption it,' 'give me a transcript.' Twelve variations of the same ask, buried across sessions, never once matching as a single obvious signal until Agnost clustered them.

The engineering behind this got gnarly fast, and it's the more interesting part of the launch, honestly. Running everything through an LLM to detect new clusters was too slow and too costly once volumes hit real scale. Standard embedding-based clustering, like HDBSCAN, chokes on pairwise comparisons once you're past a certain message count. So the team built a pipeline that first segments conversations by cosine drift, compresses the candidate space with BIRCH, and only then runs HDBSCAN-style clustering on what's left. LLMs get reserved for ambiguous edge cases, not the bulk of the work. On the storage side, ClickHouse needed heavy tuning around sort keys and materialized views, because pulling the last 50 events across all conversations and pulling every event within one conversation are fundamentally different query shapes.

Agnost is explicitly positioning itself away from the observability and eval crowd. Datadog-style tools tell you what technically happened; evals check cases you already anticipated. Agnost's bet is that the real gap is upstream of both: figuring out what users wanted before you knew to test for it. The company is already processing around a million messages a day across multiple paying customers, pricing starts free and tops out at $499 a month for Pro, and they're SOC 2 Type 1 compliant with Type 2 in the works.