SPARSEUP Brings Apache-2.0 Sparse Retrieval to ModernBERT
Linkup Research has released SPARSEUP, an Apache-2.0 learned sparse retrieval model built from the ModernBERT family and trained as a SPLADE-style encoder. Linkup reports 56.4 average nDCG@10 on BEIR-13, while its published Seismic experiment reaches more than 97% recall on MS MARCO at roughly 380 microseconds average query latency.
The release targets search and retrieval systems that want neural relevance while retaining sparse, token-addressable representations compatible with inverted-index infrastructure. The model is available on Hugging Face and supports Transformers and Sentence Transformers integration through custom model code.
A parameter-count detail needs precise wording. Linkup describes SPARSEUP around a 149M-parameter ModernBERT backbone, while Hugging Face currently reports 188.4M parameters for the hosted repository artifact. The larger repository count is consistent with SPARSEUP restoring an MLM/output head on top of the LateOn-derived backbone. Deployment sizing should therefore use the actual model artifact instead of treating the 149M backbone figure as the complete serialized parameter count.
SPARSEUP specifications
| Item | Published detail |
|---|---|
| Architecture | Learned sparse / SPLADE-style retrieval encoder |
| Backbone | ModernBERT family; Linkup describes a 149M backbone |
| Hosted artifact | Hugging Face reports 188.4M parameters |
| Base checkpoint | lightonai/LateOn-unsupervised |
| Language | English |
| License | Apache 2.0 |
| BEIR-13 | 56.4 average nDCG@10 |
| Average sparsity | About 47 non-zero terms/query and 190/document |
| Sparse output space | About 34,000 dimensions after case folding |
| MS MARCO / Seismic result | >97% recall at ~380 µs average query latency |
| Frameworks | Transformers; Sentence Transformers |
Sources: Linkup technical release, Hugging Face model repository, and independent release coverage.
How the sparse representation is built
SPARSEUP starts from LightOn's LateOn-unsupervised checkpoint. Linkup restored ModernBERT's masked-language-model head and fine-tuned the resulting model with the open training mixture used around LightOn's DenseOn and LateOn work.
The encoder produces vocabulary-aligned token weights instead of one compact dense vector. Linkup uses three mechanisms to control the representation size: a logit shift of 15, a top-12 expansion cap per input position, and case folding. The resulting vectors average roughly 47 non-zero terms for a query and 190 for a document.
That structure matters operationally. Sparse outputs can be stored in an inverted index, and individual dimensions correspond to vocabulary terms. Search engineers can inspect which terms contributed to a match while retaining learned expansion beyond literal keyword matching.
The benchmark result needs two comparisons
Linkup's 56.4 BEIR-13 result is strong within the public vocabulary-based sparse encoders it compares against. Its published comparison places SPARSEUP ahead of opensearch-neural-sparse-encoding-doc-v3-gte at 54.6, ModernBERT-VT at 52.4, SPLADE-v3 at 51.7 and Granite's 30M sparse embedding model at 50.6.
A second comparison clarifies the architecture trade-off. With the same general backbone and training-data family, Linkup reports DenseOn at 57.9 and LateOn at 58.9 on BEIR-13. SPARSEUP therefore gives up some average retrieval quality in that controlled comparison while providing a sparse representation suited to inverted indexes and direct token-level inspection.
The latency result belongs to the retrieval stack as a whole. The approximately 380 µs figure is reported for querying a Seismic sparse index on MS MARCO at greater than 97% recall. It measures sparse-index serving potential; end-to-end document encoding and RAG latency require separate measurement.
Where SPARSEUP fits in a retrieval stack
SPARSEUP is most relevant where exact terminology, identifiers, product names, code symbols or domain vocabulary matter and where an existing search architecture already benefits from inverted indexes. Those properties make it a practical candidate for technical documentation, enterprise search, code-adjacent corpora and hybrid RAG retrieval.
Dense retrieval remains attractive when semantic similarity across different wording dominates. Late-interaction systems can improve relevance further at the cost of more complex storage and scoring. SPARSEUP adds a current open sparse option to that design space, making it easier to benchmark sparse, dense and late-interaction retrieval on closely related model and data foundations.
For production evaluation, the useful test is corpus-specific: measure recall or nDCG on representative queries, index size, document-encoding cost, p95/p99 query latency and throughput at the intended concurrency. Hybrid experiments should also compare SPARSEUP alone with sparse+dense fusion before replacing an established BM25 or dense pipeline.
Deployment notes
The Hugging Face repository is tagged for Transformers, Sentence Transformers and text-embeddings-inference, and it uses custom code. Teams that load the model with trust_remote_code=True should pin a reviewed repository revision as part of normal model-supply-chain controls.
The Apache-2.0 license permits broad commercial and research use. The current model card identifies English as the supported language, so multilingual retrieval should be validated separately or handled with a model designed for those languages.
Bottom line
SPARSEUP is a useful open release because it combines modern neural retrieval training with an operationally familiar sparse representation. Its 56.4 BEIR-13 score trails the related DenseOn and LateOn results, while its sparse output enables inverted-index serving, token-level interpretability and a reported sub-millisecond Seismic query path. For teams building search or RAG systems, SPARSEUP is a credible benchmark candidate alongside BM25, dense embeddings and late-interaction retrieval, with enough implementation detail to support production-oriented evaluation.