How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code

Three months after restarting Papers with Code, Hugging Face has turned the site’s search into a careful split between batch work and live queries. That split is the whole trick: the expensive embedding jobs run offline, while the tiny piece needed for a user search sits behind an Inference Endpoint. If that endpoint is cold, busy, or unhealthy, the system drops straight back to full-text search instead of making people wait.

The team says a paper search engine has to do more than match words. It should find exact titles and arXiv IDs, but also understand messier requests like “small language models for code generation,” typos, incomplete titles, and navigational queries such as “the original BERT paper.” To handle that, Papers with Code combines PostgreSQL full-text search with pgvector and fuses the results with reciprocal rank fusion, or RRF. The result is meant to keep exact matches strong without losing the fuzzy semantic stuff.

For embeddings, the production setup uses Qwen/Qwen3-Embedding-0.6B pinned to an exact revision, with 256-dimensional, L2-normalized vectors. The input format is treated like an API contract: normalized title, two line breaks, normalized abstract. The team records the model repo, revision, output dimension, input-format version, query-versus-document mode, normalization method, and a content hash for each vector generation. That kind of bookkeeping sounds fussy, and it is. It also prevents the usual silent breakage when an embedding pipeline drifts.

The batch side runs on Hugging Face Jobs, which the team uses for full-corpus rebuilds. A PostgreSQL snapshot is exported into bounded JSONL shards, synced to a Storage Bucket, then mounted into an l4x1 Job with an NVIDIA L4 GPU. Each completed shard gets its own marker, so retries can pick up where they left off. In a 5,000-paper pilot, the Qwen job encoded about 75 papers per second at 1024 dimensions, and the same material could be deterministically produced at 512 and 256 dimensions for comparison.

Buckets sit in the middle as durable handoff storage. They hold the immutable run directories, the manifests, the checksums, and the output shards. The importer checks all of that before anything touches the search index, then builds a separate HNSW index and only marks it active once coverage is complete. For live traffic, the query path uses an authenticated Inference Endpoint backed by TEI, with a one-second timeout, validation on the returned vector, and a circuit breaker after repeated failures. On the 5,000-paper pilot, the 256-dimensional index hit 0.9955 Recall@20 against exact search, with 1.31 ms p50 and 2.21 ms p95 lookup latency, while using about 27% of the storage of the 1024-dimensional version.