GLiNER2.5-Decide: 340M Open-Weight Decision Model Runs on CPUs

By AiCybr Editorial · Published September 27, 2026

Fastino released GLiNER2.5-Decide on September 24, 2026, as a 340-million-parameter open-weight model for schema-defined classification and operational decisions. It runs locally on CPUs or GPUs through GLiNER2, uses an Apache 2.0 license, and returns constrained answers with probabilities and confidence scores instead of generating free-form text.

Fastino reports 167.3 ms p50 end-to-end latency on a 48-vCPU Intel Xeon Platinum 8581C and 38.3 ms on an NVIDIA V100 for its short-document workload. On the company's fast-decisions benchmark, the current model card reports 60.2% exact-match accuracy averaged across 17 domains with 300 held-out examples per domain.

The model targets routing, intent classification, sentiment, moderation, priority, triage and other decisions where the possible outputs are known in advance. That makes it relevant to local AI and agent systems that currently spend a larger generative model call on a small structured choice.

GLiNER2.5-Decide specifications

The Hugging Face repository currently stores a roughly 1.95 GB safetensors checkpoint. The model card loads the model with AutoExtractor.from_pretrained("fastino/GLiNER2.5-Decide") after installing gliner2.

What the model is designed to do

GLiNER2.5-Decide accepts text plus a caller-defined set of labels or typed questions. A support system, for example, can supply labels such as refund, cancellation, login problem and shipping delay. The model scores the supplied choices and returns the selected structured result.

Fastino lists operational use cases including customer and banking intent, travel and clinic requests, review sentiment, document classification, email and ticket routing, human handoff, moderation, severity, urgency and spam detection. The same model can score several decision heads in one call.

This architecture is useful when an application already knows the valid action space. A deterministic schema can reduce output parsing and constrain downstream automation to explicitly permitted choices.

Benchmark results

Fastino's current model card reports the following exact-match averages on fastino/fast-decisions. The suite covers 17 English-language domains, with 300 held-out examples per domain, and uses the same text and candidate labels for each compared model.

These figures are Fastino-run benchmark results, so production selection should include evaluation on the target application's own labels, class balance and error costs. The benchmark is especially useful for comparing zero-shot structured classification under one published protocol; it is not a general language-model reasoning benchmark.

Fastino's September 24 release article reports a 60.1% average for its unseen 17-dataset evaluation, while the current Hugging Face model card reports 60.2% on the published fast-decisions table. The article uses the current model-card figure for the comparison table and preserves the release-page number as a separately attributed measurement.

CPU deployment changes the economics of simple agent decisions

A 340M encoder model can occupy a different deployment tier from multi-billion-parameter generative models. Fastino explicitly supports local CPU execution and air-gapped deployment, which can suit high-volume routing or classification services where data locality and predictable structured outputs matter more than open-ended generation.

The published 167.3 ms CPU p50 was measured on a 48-vCPU Xeon Platinum 8581C. It should be treated as a reference measurement, not a latency promise for desktop CPUs, mini PCs or small virtual machines. Thread count, input length, batching and runtime configuration can materially change local performance.

For agent architectures, a small decision model can sit ahead of expensive generation: classify an incoming request, choose a workflow, decide whether human review is required, or score urgency before invoking another service. The strongest deployment case is a repetitive decision with a stable set of valid outputs and enough traffic to make local inference operationally worthwhile.

English and multilingual models are separate

The 340M GLiNER2.5-Decide checkpoint is the English model. Fastino separately publishes GLiNER2.5-multi-Decide, a 287M multilingual model.

On the same published English benchmark, the multilingual checkpoint scores 56.7% versus 60.2% for the 340M English model. Applications handling multilingual input should evaluate the multilingual checkpoint directly instead of extrapolating from the English result.

Practical limits

GLiNER2.5-Decide is a specialist structured-decision model. Its output space is supplied by the application, which makes it appropriate for classification and routing but leaves open-ended writing, explanation and broad reasoning to a generative model.

The benchmark also measures exact-match classification on Fastino's own evaluation suite. Accuracy on a production taxonomy can move substantially with domain language, ambiguous labels, rare classes and the number of candidate choices. Teams should build a held-out set from real traffic and measure per-class precision, recall and costly failure modes before allowing a decision to trigger consequential automation.

For workflows that require both extraction and classification, the broader GLiNER2.5 family also supports schema-driven entity, relation and record extraction. The Decide release adds a more explicit decision-oriented interface and benchmark to that ecosystem.

Deployment fit

GLiNER2.5-Decide is most compelling for local or private systems that repeatedly map text into a known action set: ticket routing, moderation queues, document classification, intent detection, agent handoff and policy-oriented triage. Its Apache 2.0 weights and CPU path make it straightforward to prototype without a hosted inference dependency.

A generative model remains the appropriate layer when the application needs synthesis, explanation, tool planning or an answer that cannot be represented by a bounded schema. In mixed systems, the two model classes can be complementary: the compact classifier handles high-frequency decisions and the generative model receives tasks that require broader language capability.

Sources

• Fastino — GLiNER2.5-Decide release, September 24, 2026: https://fastino.ai/blog/gliner-2-5-decide-open-weight-decision-model
• Fastino — GLiNER2.5-Decide model card and benchmark: https://huggingface.co/fastino/GLiNER2.5-Decide
• Fastino — GLiNER2.5-Decide checkpoint artifact: https://huggingface.co/fastino/GLiNER2.5-Decide/blob/main/model.safetensors
• Fastino — GLiNER2 open-source repository and model family: https://github.com/fastino-ai/GLiNER2
• Shattered — independent September 27 technical coverage: https://shattered.io/fastino-gliner2-5-decide-340m-cpu-model-2026/