How to Match LLM Patterns to Problems

Eugene Yan · 2 years ago

The article presents a framework for matching architectural and operational patterns to common problems encountered when building LLM applications, distinguishing between external models like ChatGPT and internally-hosted open models. Key patterns include retrieval-augmented generation (RAG) for reducing hallucinations, fine-tuning for domain-specific performance, evals for measuring improvements, guardrails for output validation, caching for latency reduction, and defensive UX for managing user expectations. Organizations choosing between external and internal LLMs face different constraints: external models offer state-of-the-art quality but have rate limits and data privacy concerns, while internal models require development and hosting costs but provide full control.