How to Build an Open-Domain Question Answering System?
Lilian Weng
The article reviews approaches for building open-domain question answering systems that can answer factual questions without relevant context provided. Key systems discussed include retriever-reader frameworks using classic IR (TF-IDF) or neural IR methods, with models like DrQA (2017) establishing Wikipedia as a standard knowledge source and top-k=5 document retrieval becoming common practice. The resulting systems enable applications like chatbots and AI assistants by decomposing QA into document retrieval and answer extraction stages.
Why it matters
[Updated on 2020-11-12: add an example on closed-book factual QA using OpenAI API (beta). A model that can answer any question with regard to factual knowledge can lead to many useful and practical applications, such as working as a chatbot or an AI assistant🤖. In this post, we will review several common approaches for building such an open-domain question answering system.
