Experimenting with LLMs to Research, Reflect, and Plan
Eugene Yan
A developer built several Discord-based tools using LLMs, including /summarize for extracting URL content, /sql and /search agents for querying databases and search APIs, and /board for synthesizing advice from public figures' writings. The tools used OpenAI's gpt-3.5-turbo and gpt-4 models with LangChain, Pinecone embeddings, and Google Search integration. The developer identified retrieval issues where embedding-based systems failed to consistently surface relevant source documents and proposed solutions including fine-tuning embeddings with triplet loss, chunking documents by sections rather than tokens, and ensembling semantic search with keyword-based methods like BM25.
Why it matters
Also, shortcomings in document retrieval and how to overcome them with search & recsys techniques.
