Building A Generative AI Platform

Chip Huyen · 2 years ago · 48

The article outlines a modular architecture for deploying generative AI applications, starting from basic model queries and progressively adding components like context augmentation, retrieval-augmented generation (RAG), guardrails, routing, caching, and observability. Key retrieval approaches include term-based methods like BM25 (faster, cheaper) and embedding-based vector search (more computationally expensive but improvable), with hybrid search combining both approaches and reranking to optimize results. The platform design enables models to access external data sources, SQL tables, and web search through read-only and write actions, with query rewriting to improve retrieval accuracy in multi-turn conversations.