CLM-8B: Open Contrastive Model Scores Agent Actions With Cached Embeddings

By AiCybr Editorial · Published September 27, 2026

Contrastive-LM released CLM-8B on September 23, 2026, as an Apache-2.0 model for scoring bounded agent actions, typed decisions and candidate solutions. The released system places two roughly 20-million-parameter projection heads over a frozen Qwen3-8B encoder and trains them with a bidirectional InfoNCE objective so states and candidate actions can be embedded separately.

That separation is the central deployment feature. Candidate-action embeddings can be cached and reused while each new state is encoded once, followed by vector scoring against the candidate set. Contrastive-LM reports up to 9× lower latency than Jev across its zero-shot computer-use, gaming and tool-calling evaluations, with the largest gains where candidate sets are large or actions repeat across states.

The reference head is about 75 MB, while deployment still requires the Qwen3-8B encoder. The official package requires Python 3.10+, Linux and an NVIDIA GPU for the documented vLLM setup. The default serving example uses a 2,048-token model limit; longer states require raising the limits on both the vLLM encoder and CLM server, with additional GPU-memory cost.

CLM-8B specifications

How CLM scores decisions

CLM maps a state and its candidate actions into a shared embedding space. At inference time, the state encoder produces one vector for the current context. Candidate actions are encoded separately, and their vectors can remain cached when the available action set stays stable. Dot products between the state and action vectors provide the scores used for ranking or a softmax probability distribution.

The package exposes three typed question forms: Noul for a truth probability, Choice for selection among declared options and Score for an ordered rubric. Its lower-level ranking interface can score free-form candidates such as tool names, next actions or best-of-N generated solutions.

This architecture fits agent loops where the expensive reasoning or generation step has already produced a bounded set of possible actions. Repeated tool menus are particularly favorable because their action embeddings can be reused across requests.

Training recipe

The project documents three training stages:

1. roughly 60 million Nemotron DQA question-answer pairs for broad state-action alignment;
2. roughly 30 million synthetic hard negatives for finer discrimination among plausible alternatives;
3. roughly 1 million agentic trajectories for post-training on operational decisions.

The released reference model uses Qwen3-8B last-token-pooled embeddings. The project describes the head as encoder-locked, so a head trained for this representation must be paired with the matching encoder and pooling setup.

Reported benchmark results

Contrastive-LM's zero-shot evaluation compares CLM-8B with TypeSafe's Jev across computer-use, gaming and tool-calling tasks. The maintainers report performance broadly on par with Jev and latency up to 9× lower. The speed advantage is workload-dependent: cached, repeated actions and large candidate pools create more opportunity to reuse embeddings.

The project also reports stronger coding-agent verifier results after task-specific fine-tuning. A fine-tuned CLM head selected among candidate solutions on 38 held-out DeepSWE tasks and 30 held-out Terminal-Bench 2.1 tasks. It reached 81.6% on DeepSWE and 87.6% on Terminal-Bench 2.1, while the maintainers report 4.1–5.7× lower latency than Jev on an H100 in those experiments.

Those verifier figures are held-out subset evaluations with fine-tuned heads, not zero-shot scores from the reference checkpoint or full public-leaderboard submissions. They are most useful as evidence that the contrastive architecture can be specialized for best-of-N selection; production teams should reproduce accuracy and latency on their own action distributions and hardware.

Where caching changes the economics

A generative model normally spends compute decoding output tokens even when the final decision is one of a small number of known choices. CLM converts that workload into embedding plus similarity operations. When candidate actions stay unchanged, their embeddings can be calculated once and reused.

The model card reports a 13× latency advantage over Jev at roughly 1,000 candidates in the project's measurements. Smaller or constantly changing candidate sets reduce the caching advantage. The design therefore maps most naturally to tool routing, retrieval ranking, game or UI actions, triage, policy selection and verifier stages with stable option sets.

Deployment fit and limits

CLM-8B is a scorer for supplied candidates. It does not synthesize a missing action, so the surrounding agent must construct a sufficiently complete candidate set. Its probabilities are relative to those supplied choices.

The reference deployment also remains an 8B-class GPU workload because the 75 MB head sits on Qwen3-8B. The small head lowers fine-tuning and model-specific storage costs; it does not turn the full serving stack into a 75 MB model.

For teams already running Qwen3-8B through vLLM, CLM provides a relatively compact additional decision layer with a TypeSafe-compatible API and reproducible training/evaluation code. Its strongest deployment case is repeated bounded decision-making where action embeddings can stay hot in cache and where latency matters enough to justify a specialized scorer.

Bottom line

CLM-8B offers a distinct open approach to agent decision latency: separate the changing state from reusable actions, encode both contrastively and replace token generation with cached vector scoring. The released Apache-2.0 artifacts make the approach directly testable, while the published 9× and verifier results remain project-reported measurements tied to specific tasks, candidate sets and hardware.

The practical evaluation question is therefore workload fit. Agents with stable tools, repeated action menus or best-of-N verifier stages can exploit the architecture directly; workloads that require open-ended action generation still need a generative model elsewhere in the loop.

Sources

• Contrastive-LM CLM repository and reproducible evaluation code: https://github.com/Contrastive-LM/CLM
• CLM-v0.1-8B model card, requirements, benchmark notes and Apache-2.0 weights: https://huggingface.co/Contrastive-LM/CLM-v0.1-8B
• contrastive-lm package release: https://pypi.org/project/contrastive-lm/
• Independent technical coverage of the release and benchmark scope: https://www.marktechpost.com/2026/09/23/contrastive-lm-releases-clm-8b-an-open-system-one-model-that-scores-agent-actions-up-to-9x-faster-than-jev/