General-Purpose Decision Models and Fast "System 1" Inference

A new model category is emerging between task-specific classifiers and full generative LLMs, fast, general-purpose decision models. TypeSafe AI recently pioneered this approach with Jev, which they call a “System 1” decision-making AI model designed to evaluate system states and return fast, structured, typed probabilistic answers.¹

Instead of training a separate classifier for every task, or asking a large language model to reason and generate text for every decision, a decision model accepts the input plus a runtime-defined decision space and returns a structured prediction, usually with probabilities.¹

This creates a reusable inference layer for classification, routing, scoring, ranking, policy selection, safety checks, and agent decisions.

input state
    +
runtime-defined choices / criteria
    ↓
general-purpose decision model
    ↓
structured decision + probabilities

The pitch is that many workloads that are currently handled either by bespoke classifiers or expensive LLM calls may eventually be handled by small, fast, general-purpose prediction models.^{1 2}

Classifiers and LLMs

Production systems currently tend to choose among rules, specialized classifiers, and LLMs, where each comes with different flexibility-cost trade-offs.

A conventional classifier has a fixed task encoded into its weights. A fraud classifier predicts fraud, an intent classifier predicts intent, and a support router predicts queues. Changing the task usually means collecting data and training another model. A general-purpose decision model instead receives the task at inference time. An application might provide four categories such as billing, technical, account, and other, along with descriptions of what each means. The same model can later be given an entirely different decision space without retraining.^{1 3}

This is the central architectural difference:

Jev and the “System 1” idea

System 1 thinking is our brain’s fast, automatic, unconscious, and emotional response to situations and stimuli, which was popularized by psychologist Daniel Kahneman in his book Thinking, Fast and Slow. This system handles routine snap judgments without needing much conscious effort or energy.⁴

TypeSafe AI positions Jev as a “System 1” model. Rather than generating arbitrary language token by token, it is designed to take a state and answer structured questions about that state.¹ The idea is that many AI workloads do not actually require generation, but rather structured decision-making. Applications frequently need answers to questions such as: Which queue should receive this ticket? Is this request suspicious? Which tool should an agent invoke? How risky is this action? Which policy applies? A full LLM can answer all of these, but text generation is incidental to the real task. The desired output is a decision.

That suggests a model stack where different model families specialize by computational role rather than by domain.

rules
  ↓
specialized classifiers
  ↓
general-purpose decision models
  ↓
reasoning / generative LLMs

The higher layers offer greater flexibility and reasoning capacity, while the lower layers should generally be cheaper and faster.

A recent Red Hat benchmark

Red Hat compared Jev and other decision models against specialized classifiers and LLM-based guardrails on prompt-injection detection and content safety.²

For prompt-injection detection:

For content safety:

The benchmark does not show that decision models are universally better. Unsurprisingly, Red Hat’s DeBERTa model came within roughly 0.3 percentage points of the 35B Qwen model on prompt-injection detection while running much faster.² When a problem is stable, its labels are known, and sufficient training data exists, a classifier specifically optimized for that task is extremely difficult to beat. However, decision models may be more useful for handling tasks where the taxonomy changes frequently, labeled data does not yet exist, or the cost of maintaining many separate ML pipelines becomes significant.

Economics

A conventional ML workflow may require defining labels, gathering examples, labeling data, training models, evaluating them, and deploying them. That process is reasonable for a stable, high-volume task, but expensive for rapidly evolving application logic. A general-purpose decision model could potentially begin solving the same problem as soon as the categories and their meanings are defined. This changes the economics of classification. The system gains a third option: use a reusable model that understands arbitrary decision spaces at runtime.

Calibrated Probabilities

Decision models often expose probabilities across the available choices so that downstream systems can reason about uncertainty rather than merely receiving a categorical answer.^{1 3} For example, a system might automatically execute decisions above a high-confidence threshold, escalate ambiguous cases to a stronger model, and send extremely uncertain or high-risk cases to human review. This enables cascaded architectures where expensive intelligence is used only when necessary.

However, emitting a number such as 0.9 does not mean the model is genuinely 90% likely to be correct. For probabilities to be operationally meaningful, they need to be calibrated. Among predictions assigned approximately 80% probability, roughly 80% should actually be correct.⁵ Calibration therefore needs to be measured independently alongside accuracy, precision, recall, latency, throughput, cost, robustness, and out-of-distribution behavior.

Decision models also reduce the failure mode of invalid outputs in generative systems. If an application declares three valid decisions, the model can be constrained to return only those decisions. It cannot wander and invent categories.^{1 2}

Local decision models

This paradigm is already extending beyond Jev. Ollama already introduced a /v1/systemone interface for Jev-style decision models, where applications provide a state, one or more questions, and runtime-defined criteria.³ Local models such as Nimble and Tev1 can therefore be used as general-purpose classifiers without training a model for every taxonomy. The same model can classify one request into support queues, another into safety categories, and another into agent actions.³

Decision models in production

Practical architectures are likely to use all of these techniques together.

A mature system can also escalate based on uncertainty.

request
   ↓
rules / cheap classifier
   ↓
general decision model
   ├── confident → execute
   └── uncertain
          ↓
      stronger LLM
          ├── resolved → execute
          └── unresolved → human

Instead of spending the same amount of computation on every request, the system allocates intelligence according to difficulty.

Broader significance

Foundation LLMs demonstrated that a single model can generalize across many generative tasks. General-purpose decision models are attempting something analogous for predictive tasks. Typically, in the past, prediction has often meant one model per problem. Fraud detection, intent classification, spam detection, routing, moderation, and risk assessment each received their own datasets, training pipelines, deployments, and monitoring. The emerging alternative is one broadly trained predictive model plus a runtime-defined task.

If this approach works well enough, many application architectures may shift from maintaining large numbers of tiny bespoke models or invoking full LLMs for trivial decisions toward reusable inference services optimized for particular computational roles. The model ecosystem would then look less uniform and more like a collection of general-purpose model classes:

Bottom line

The key idea behind Jev-style models is a different deployment model for prediction. Instead of training a dedicated model for every classification problem, applications can define the decision space at runtime and use a reusable model to make fast structured predictions. That makes decision models particularly attractive for dynamic tasks where labels change frequently, training data is limited, or calling a full LLM would be unnecessarily expensive.

Specialized classifiers will remain superior for many stable, data-rich problems, while reasoning LLMs will remain necessary for genuinely complex inference. The interesting opportunity is the large middle layer between them: semantic decisions that require general understanding but not full generative reasoning.

If these models improve in speed, calibration, and accuracy, they could become a standard inference primitive alongside embeddings, rerankers, and LLMs.