Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU

Fastino Labs has put out GLiNER2.5-Decide, a 340M-parameter open-weight decision model built for structured judgment calls. It takes text plus a schema of typed questions and returns answers with probabilities, confidence scores, and feasibility metadata. The pitch is not that it writes better prose. It is that it makes the annoying little decisions inside agent pipelines less annoying.

The model is a non-generative classifier based on a DeBERTa-v3-large encoder and fine-tuned from gliner2-large-v1. There are no generated tokens and no prompt template to manage. Instead, label sets arrive at call time, and each question in the schema can define the allowed answers, whether it expects one or many, and whether the answer has an order. Schemas can also include instructions, examples, label descriptions, and rules that connect answers across questions.

That last part is the interesting bit. Fastino says the system uses joint decoding: the encoder scores permitted answers, then a constrained decoder searches for the highest-scoring combination that still obeys the declared rules. So if one field says “unsafe” should follow from any detected harm, the model can return a paired result like safety=unsafe and harm_type=prompt_injection instead of two conflicting labels. Fastino is also clear about limits. This model is for operational decisions, not open-ended reasoning or explanations.

Fastino tested GLiNER2.5-Decide on an internal suite called Fast Decisions, with 5,100 examples across 17 datasets. It scored 60.1% average exact-match accuracy and led 9 of the 17 datasets. The strongest results were in intent routing, where it hit 75.3% on support intent and 64.3% on banking intent. Fastino says the model runs under Apache 2.0 with pip install gliner2, works on CPU, GPU, or in air-gapped environments, and is also available through hosted inference and fine-tuning via its GLiNER API.

Fastino also says the checkpoint is practical on real hardware. At 64 tokens, p50 latency was 167.3 ms on a 48-vCPU Intel Xeon Platinum 8581C, and between 38.3 ms and 47.3 ms on the GPUs it tested. At 1,024 tokens, the A100 was fastest at 52.6 ms, ahead of the V100 at 75.6 ms and the L4 at 131.4 ms. The company has also published a 1B version and a multilingual variant, but the main story here is simpler: smallish model, strict structure, and enough speed to sit inside production workflows.