Perplexity Releases pplx-embed-v2-context-9b-preview: A Contextual Embedding Model That Retrieves Answers and Their Supporting Evidence

Perplexity Research and turbopuffer have put out pplx-embed-v2-context-9b-preview, a contextual embedding model built for RAG systems. The twist is simple but pretty important: it doesn’t just hunt for one “gold” passage. It tries to bring back the answer and the surrounding evidence needed to check it.

That matters because chunked documents are messy. A sentence that answers the query often leans on a heading, an entity, or a definition sitting somewhere else in the file. Traditional training makes that worse by turning every non-selected chunk into a negative, even when those chunks contain the bits that make the answer trustworthy.

Perplexity’s answer is a teacher-student setup. Its query-aware context compression model reads the query and the whole document together, scores every token, then rolls those scores up into chunk scores using the mean of the top n tokens in each chunk. Those scores become a soft target through a temperature-scaled softmax. The student is trained with forward KL divergence against that distribution, plus an InfoNCE document loss where a document scores by its best chunk, borrowing the MaxSim idea from ColBERT.

The training also samples different chunking strategies each batch, which is the sort of detail that sounds minor until you remember how often retrieval systems fall apart when the chunk boundaries move. The model starts from an in-house 9B ColBERT retrieval model, projects to 2048 dimensions, and also supports 1024 dimensions through Matryoshka training. Perplexity says quantization-aware training enables native int8 embeddings, and the release combines several checkpoints into a model soup.

As for shipping: yes, but only as a self-hosted preview. The weights are on Hugging Face under the MIT license, loading needs transformers>=5.4.0 with trust_remote_code=True, and it is not yet on the Perplexity API. The model card also warns that the weights and interface may change without backward compatibility. Training used roughly 430 datasets covering more than 50 languages, with no ConTEB data.