AnyJev Turns Open LLMs Into Calibrated Decision Models Without Fine-Tuning

By AiCybr Editorial · Published September 27, 2026

Nokia Applied Research has released AnyJev, an Apache-2.0 Python library that converts open language models into typed decision systems for classification, routing and scoring workloads. The current project supports Hugging Face Transformers and vLLM, with decision levels ranging from zero-label bias correction to small closed-form heads trained on a few hundred labels.

The project targets a specific production problem: applications often need a bounded decision and a probability that can drive an automation threshold. AnyJev reads model outputs or hidden states to return distributions over defined choices, yes/no decisions or ordered scores.

On the project's Qwen3-8B / BANKING77 20-way benchmark, raw label-token logits reached 74.7% accuracy, a 23.0% option-order flip rate and 0.240 expected calibration error (ECE). AnyJev's zero-label L0 method reported 80.3% accuracy, a 7.3% flip rate and 0.184 ECE. L1, which adds temperature scaling using 100–500 labels per question, reported 80.7% accuracy and 0.095 ECE. These are project-reported measurements from 300 test items and should be validated on the target workload before production use.

What AnyJev changes

AnyJev exposes four operating levels with different data and serving requirements.

For a K-option choice, L0 evaluates cyclic rotations so each option occupies each label position. The distributions are combined to reduce position bias, then adjusted using a prior estimated from observed traffic. This increases inference work relative to a single raw-logit readout because L0 requires K prefills for K choices.

L2 uses a different path. It fits a closed-form head on hidden states from an intermediate model layer. The base model weights remain unchanged, while each question receives a small fitted head. AnyJev's current documentation says L2 heads require 100–300 labels per question and are specific to the fitted question and model.

Measured decision quality

The project's BANKING77 experiment illustrates why calibration can matter as much as top-line accuracy for automation.

The final metric is particularly relevant to routing systems. A classifier can have useful aggregate accuracy while producing confidence values that are poorly suited to a fixed automation threshold. In AnyJev's experiment, calibration increased the share of requests meeting its ≤5% measured-error threshold from 7.7% to 52.0%.

The sample is small enough that teams should reproduce the measurement with their own class balance, prompts and failure costs. AnyJev's repository explicitly notes that its 5% risk coverage estimate is high variance at n=300.

L2 can use less than the full model depth

AnyJev also reports L2 experiments across five Qwen3 models using 20 typed-decision questions and 2,000 held-out decisions. The project fits its head to intermediate hidden states and can truncate the model before the final transformer blocks.

Those numbers come from the AnyJev maintainers and provide evidence for evaluating the technique under the documented benchmark configuration. Model architecture, serving engine, question definition and label distribution can materially change the result.

Deployment with Transformers and vLLM

The package requires Python 3.10 or newer and is distributed on PyPI. The repository currently carries an Apache-2.0 license.

For L0/L1, AnyJev can work with generation-style serving because it needs the model's label-token probabilities. L2 needs hidden states. The project's vLLM deployment path therefore uses an embedding/pooling server and reads the selected hidden representation through the pooler.

The maintainers also provide a pipeline that truncates a model, launches the server, fits a head and measures accuracy, ECE and decision latency against a full-depth baseline. A production evaluation should run that measurement on the target hardware and dataset to establish whether the repository's Qwen results transfer.

Where it fits

AnyJev is most relevant when the output space is known in advance: support-ticket routing, moderation categories, agent tool selection, escalation decisions, policy classification or bounded quality scores. Those workloads benefit from an explicit probability distribution and an abstention or human-review threshold.

Open-ended generation remains a separate workload. The repository also records several concrete boundaries: L2 heads are per question and per model; the current letter-token readout supports at most 26 options; some tasks such as the project's maze-edge and Minesweeper tests failed to beat a trivial baseline; and L0's prior correction can reduce accuracy when one label dominates heavily.

These constraints make workload-level validation part of deployment, especially when the resulting probability controls an irreversible action.

Bottom line

AnyJev provides an inspectable path from a general open LLM to a bounded decision service. L0 offers a zero-label method for reducing option-position bias, L1 adds lightweight probability calibration, and L2 trades a few hundred labels for a small closed-form head that can operate on an intermediate model layer.

Its clearest deployment role is separating high-volume, schema-bounded decisions from generative work and measuring whether calibrated probabilities allow more requests to be automated at an acceptable error rate. The project exposes benchmark data, code and reproducible artifacts for teams to test that proposition on their own workloads.

Sources

• Nokia Applied Research — AnyJev repository and benchmark documentation
• AnyJev benchmark results
• AnyJev L2 / early-exit results
• AnyJev on PyPI
• MarkTechPost — Nokia open-sources AnyJev