Linkup Research Releases SPARSEUP: A 149M-Parameter Open-Source Sparse Embedding Model
MarkTechPost Asif Razzaq
Linkup released SPARSEUP, an open sparse embedding model with 149M parameters. It hit 56.4 on BEIR-13 and claims a rare public spot under 150M.
Based on reporting by MarkTechPost, Asif Razzaq — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Linkup Research has put out SPARSEUP, an open-source sparse embedding model built on a 149M-parameter ModernBERT backbone and released under Apache 2.0. The weights are on Hugging Face, and the model can be loaded through Transformers or Sentence Transformers with trust_remote_code=True.
The pitch is simple: most retrievers today are dense, while sparse models write weights over vocabulary terms. That makes them easier to slot into inverted indexes and, at least in theory, easier for people to inspect. Linkup says it was pushed to fill the sparse gap after LightOn released DenseOn and LateOn, which gave the field open data, a recipe, and two other retrieval styles built on the same backbone family.
SPARSEUP starts from LateOn-unsupervised, then gets its missing MLM head restored from ModernBERT’s original setup. Fine-tuning uses LightOn’s mixture with contrastive learning only, 7 hard negatives per query, plus in-batch negatives. There is no cross-encoder distillation, and the training fits on a single H100. Linkup says a plain SPLADE setup on this backbone produced bags crowded with stopwords, so it added three fixes: a logit shift, per-position top-k pruning, and case folding.
Those changes matter in practice. The model’s output dimensions fall from about 50k to about 34k. Queries and documents use [Q] and [D] prefixes, scoring is a dot product, and evaluation uses max lengths of 128 tokens for queries and 512 for documents. On BEIR-13, SPARSEUP reaches 56.4 average nDCG@10, which Linkup calls the strongest public vocabulary-based sparse encoder it knows of under 150M parameters.
The comparison gets more mixed when the backbone and data are held constant. In that setup, LateOn scores 58.9, DenseOn 57.9, and SPARSEUP 56.4. Linkup says SPARSEUP does win some tasks, including ArguAna and Touché, and beats DenseOn on HotpotQA, but it trails on more semantic sets such as FiQA and DBPedia. It also uses approximate Seismic search, while LightOn reports exact search. On MS MARCO, SPARSEUP averages 47 non-zero terms per query and 190 per document, and Linkup says Seismic gets over 97% recall in about 380 microseconds per query, single-threaded.
My take — AI-written commentary, not fact-checked reporting
This is the sort of release that actually moves the open-model conversation: not bigger, just cleaner and more usable. The nice part is Linkup didn’t hide behind one shiny score; it showed where sparse retrieval helps and where it still loses to denser cousins. That honesty is rare, and so is refusing to turn every extra dimension into a victory lap.
Read more about this at: MarkTechPost