Skip to main content

​
Overview

A decision model reads a piece of data — the state — and answers the questions you ask about it with probabilities, not prose. Ask whether a ticket is urgent and you get 0.92; ask which team should own it and you get technical, along with a probability for every team. There is no text to parse, so the answer can drive a branch, a route, or a threshold check directly. Decision models are a third model kind next to Chat and Embedding. They are fast (typically a few hundred milliseconds), cheap, and answer several questions about the same state in a single call. Use a decision model when the result is a judgement — classify, route, score, gate. Use a chat model (Agent.invoke) when you need text — summaries, replies, extraction into free-form fields.

​
Question Types

Every question has a name (the key its answer is returned under), a type, instructions, and — depending on the type — criteria.
The answers come back keyed by question name:
A score is an expected value, so it can fall between levels (e.g. 1.4). confidence describes how concentrated the probability distribution is — it is not the probability of the chosen option.

​
State

The state is what the questions are about. It can be a string, a JSON object, or a JSON array — for example a chat transcript as a list of {"role", "content"} messages. Structured state usually gives better answers than one concatenated string:

​
Supported Providers

Decision models are used through a Model Account, like chat models: Anthropic, AWS Bedrock and Gemini do not offer decision models.

​
Setting Up a Decision Model

  1. Create a Model Account for TypeSafe, OpenRouter or OpenAI.
  2. Open the account. Discovered decision models are listed with the kind Decision, disabled by default.
  3. Enable the models you want to use. For an OpenAI account, add the model (or edit a discovered one) and set its kind to Decision.
A decision call fails with “Model is not available on this model account” when the model is disabled, missing from the account, or not of kind Decision.

​
Using Decision Models

Unlike chat calls, a decision call always names its model — there is no organisation default. The model account is optional and defaults to your organisation’s AI settings account, which is usually a chat account, so in practice you pass both.

​
Usage and Billing

Every decision call is recorded with its input and output token counts and attributed to the action, pipeline or flow that made it, so it shows up in your organisation’s AI usage like any other model call.

​
Tips

  • Tune thresholds on real data. Start with a named constant such as URGENT_THRESHOLD = 0.8, run a handful of real and edge-case inputs, and adjust. Probabilities are calibrated differently by different models, so do not reuse a threshold across models.
  • Add a “none” option to choices when the options may not cover every input — otherwise the model has to pick one of them.
  • Ask about the fact, not the text. “Is the customer blocked from using the product?” is answered more reliably than “Does the message mention being blocked?”.
  • Batch questions. Questions about the same state go in one call; there is no need to call once per question.