Obsidian-Copilot: An Assistant for Writing & Reflecting

Eugene Yan spent some downtime building a tool he calls Obsidian-Copilot, and the result is a useful, unglamorous window into what retrieval-augmented generation actually takes to build well. The idea is simple: point it at a section header in your notes app and it drafts a few paragraphs, pulling context from your own past writing. Point it at a daily journal and it'll help you reflect on the week gone by and plan the one ahead. No foundation model was trained here. This is entirely about plumbing — chunking, indexing, retrieval — done thoughtfully.

The chunking decision alone says a lot about how messy real-world RAG work gets. Yan initially tried the textbook approach of splitting documents into 1,500-to-3,000-token blocks, and it didn't work. His notes are mostly bullet points, so he switched to chunking by top-level bullet plus sub-bullets, landing on chunks that behave more like natural paragraphs, usually five to ten sub-bullets deep. It's a small design choice, but it's the kind of thing that separates a RAG system that returns garbage from one that returns something usable.

On search, Yan didn't bet everything on embeddings. He layered OpenSearch's BM25 keyword search underneath a semantic index, having learned from an earlier project that vector search alone leaves gaps. He tuned field boosts by hand — titles count five times more, chunk headers twice as much — and applied text-cleaning filters like stemming and stopword removal only to text fields, leaving file paths untouched. For the embedding model, he skipped the obvious choice of OpenAI's text-embedding-ada-002 and picked e5-small-v2 instead, a model ranked seventh on the MTEB leaderboard with a notably compact 384-dimension embedding size, which kept storage and compute light given his short chunks and 512-token limit.

The whole system runs as a FastAPI retrieval service alongside an OpenSearch container, wired together with Docker Compose, and surfaces inside Obsidian through a TypeScript plugin that streams generated text and links back to source notes. Generation currently runs on gpt-3.5-turbo, chosen for its speed-to-cost ratio, though Yan says he's eager to try Claude's 100k-token context window so he can feed in whole documents instead of chunks — trading retrieval precision for a model that has to do more of the filtering itself.

What's more interesting than the tool itself is where Yan thinks this pattern goes next. He suspects Microsoft, Atlassian, and Notion are quietly building versions of this aimed at team wikis, product docs, and codebases rather than personal journals. And in his own field of recommendation systems, he expects LLM-generation layered on top of existing search products to reach production before the year is out. Obsidian-Copilot, in that light, reads less like a personal hobby project and more like a rough sketch of where enterprise search tools are headed.