# How to Build a Decision Model Like Jev

How TypeSafe's Jev likely works: one-token answers, calibrated probabilities, fast serving, and the clues that it's built on Qwen 3.5.

By Jeremy Greer and Matthew Persons · October 19, 2026 · https://oumi.ai/blog/how-to-build-a-decision-model-like-jev

In our last post, [Jev vs. Fine-Tuning on BANKING77](https://oumi.ai/blog/jev-vs-fine-tuning-banking77), a 4B model fine-tuned in Oumi matched Jev's accuracy without a retrieval step. Many wonder what kind of model Jev is and whether they could build one themselves.

Turns out, you can! A decision model is really just an ordinary LLM with two changes:

1. **Training:** It learns to answer by scoring a fixed set of tokens at one spot in the prompt. It writes no text.
2. **Serving:** It reads one document once and answers every question about it in a single forward pass.

TypeSafe hasn't published how Jev works, so consider this our best guess at its secrets. We'll walk through the details, token by token, with interactive demos along the way:

- [Part 1: Training](#part-1-training): how a model answers with one token, and how to train and calibrate it
- [Part 2: Serving](#part-2-serving): how to answer many questions about one document in a single pass
- [Part 3: What Is Jev Built On?](#part-3-built-on): what Jev's answers and token counts say about where it came from
- [Part 4: How They Likely Built It](#part-4-how-built): the vocabulary swap and RLCD

## Decision Models Explained

[Jev](https://docs.typesafe.ai/models), from TypeSafe, takes a **state** (a document, ticket, JSON record, etc.) and a set of **questions** about it. Each question has a type:

- **Choice:** pick one of up to 255 options (multiple choice question, single answer)
- **Noul:** yes or no, as a probability (binary)
- **Score:** a rating on 2 to 10 levels (1-5 stars, letter grades, etc.)

Jev returns a probability for every option. TypeSafe's docs encourage putting every question you need into one request, since adding questions usually has little effect on the response time.

Chat models write their answer one token at a time and you parse the text (decoding). To get a probability, you can either ask the model to write the probability out as a number (it's bad at that), or **you can read the probability of the first answer token it would write**. The latter is what decision models build upon.

## Part 1: Training

### Tokens and Embeddings

Before getting into training, it helps to know how text gets into a model and how answers come back out. We'll use Qwen 3.5 for the examples, since [Part 3](#part-3-built-on) shows Jev is likely built on it.

A model doesn't read letters. A **tokenizer** first cuts text into pieces called tokens (a word, part of a word, a digit, a space) and gives each one a number, its ID. Qwen 3.5 knows about 248k of them.

Each ID points to a row in an **embedding table**: a list of numbers (e.g. 2,048 of them in Qwen3.5-35B-A3B) the model learned in pre-training. The first thing the model does is swap every token for its row. Tokens with similar semantic meaning end up with similar rows, so "cat" sits near "dog", and "A" near "B".

At the other end, an **output head** turns the model's final numbers (the list it ends with at each position, after all its layers) back into tokens. It's another table with one row per token. The model scores every row by how well it lines up with its final numbers, and that score is the token's logit. The highest logit is the token it would write next. Some models reuse one table for both ends, while others, like this Qwen model, keep two.

That's all a decision model needs. It adds a few new tokens to stand for the answer options (more on these in [Reserved Tokens](#reserved-option-tokens)), each with its own row in both tables. Reading an answer just means reading the output head's scores for those rows.

![Embedding demo: text cut into tokens, each token's ID pointing to a row of numbers, and the output head scoring option tokens](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/embedding-demo.png)

### The Readout Token

At every position, an LLM gives a score, called a *logit*, to every token in its vocabulary (a few hundred thousand tokens in a modern one). Normally, you sample the next token from the scores at the last position. This process is called decoding, and it's generally slow. It's often why "tokens per second" and "time to first token" are so important when discussing LLM performance.

A decision model does the same thing, but we effectively stop at 1 output token:

1. End every prompt with a special **readout token**. This is the one position where the model answers.
2. At that position, keep only the logits of the tokens that stand for the options. Ignore the rest.
3. Take the softmax of what's left (it turns scores into probabilities that add up to 1). That gives one probability per option.
4. Train with cross-entropy against the right option.

Nothing gets generated; instead you're just asking "what's the probability of each option."

### Reserved Tokens

How should we mark options? First instinct is letters: label the options A, B, C and read the logits of "A", "B" and "C". Easy! However, these have some problems.

- **Capacity:** There are only 26 letters, and Jev accepts up to 255 options. "AA" or "A1" is two tokens, which can't be read at one position.
- **Baggage:** "A" is a word, a grade, and the first answer on every multiple-choice test in pre-training. All of that leaks into the probabilities. It'd be cleaner if we could keep option tokens separate.

Instead, reserve a block of tokens, one per option slot: `<opt_0>`, `<opt_1>`, up to `<opt_254>`. Most tokenizers have spare slots that pre-training never used (Gemma's are `<unused0>`, `<unused1>`, etc.). Alternatively, increase the vocabulary by adding net-new tokens, but this requires modifying many parts of the model (tokenizer, output heads, embedding tables, etc.). These tokens start without meaning, but you can initialize them by copying the letter rows and modifying them so that they're "different enough" to not get confused. Fine-tuning will help them evolve beyond that.

Three choices in the setup matter:

- **Position:** Most models have causal attention (it goes left to right only). For example, a tag after "billing: Payments" can "see" the whole option, while a tag before cannot. This is counterintuitive but it aligns with how LLMs are typically trained.
- **Initialize:** Starting from the letter rows means the model can lean on what it already knows about answer labels from the very first training step, instead of learning from noise.
- **Shuffle:** If the right answer is usually listed first, the model learns to just pick `<opt_0>`. Shuffling the options helps prevent positional bias.

The demo below walks through one training example, starting from a real Jev request in JSON. Press **Train a step** to watch the cross-entropy loss move probability onto the right option.

![Training demo: the prompt as token chips, a reserved option token after each option, and the readout token at the end](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/training-demo.png)

### Yes/No and Scores

The same trick works for the other question types:

- **Yes/No:** Reserve two tokens, one for yes and one for no. The probability of yes is the softmax over just that pair.
- **Score:** Reserve one token per level, tied to the level's rank (the lowest level always gets the same token, whatever order the levels are shown in). Train with cross-entropy plus a penalty that grows the further an answer lands from the right level (a ranked probability score). If the answer is 5, a 4 is a much better miss than a 1!

### Training the Network

It's tempting to freeze the LLM and train a small classifier on top of its last hidden state (a "probe"). In our experience, **reading the model's own output logits at the option tokens works better**. It reuses the output layer the model already has, and the option tokens live in the same space as every word the model knows. The model does need to learn which tag belongs to which option, either with full fine-tuning or LoRA across all of its layers.

### Calibration

A decision model's output may get used as a number: "send this ticket to billing if p > 0.9." That only works if the probabilities are calibrated (of all the answers given at 0.9, about 90% should be right).

Fine-tuned models tend to be overconfident or underconfident by a fairly steady amount. The fix is straightforward: **temperature**, a single number you divide the logits by before the softmax. Above 1 spreads the probabilities out (less sure), below 1 sharpens them (more sure). After training, try a range of temperatures on held-out data and keep the one where confidence best matches how often the model is right. Do this separately for Choice, Yes/No and Score questions, since each drifts by its own amount. The winning option stays the same, so only the reported confidence changes.

One gotcha: score each temperature the same way your answers will be judged. Some benchmarks count a yes/no answer as wrong unless it's below 0.2 or above 0.8 (looking at you JevBench). The usual fit, which maximizes log-likelihood, pushes unsure answers toward the middle and loses those points. If you know the scoring rule, score your held-out answers with it at each temperature and keep the best.

It's also worth checking calibration per type of task, not only overall. An outside study, [Just Ask Jev](https://arxiv.org/abs/2609.29429), finds Jev's yes/no answers well calibrated when pooled across benchmarks, but not benchmark by benchmark, where the typical error is more than twice what sample size alone would explain. Length matters too. In our test with documents from short to very long, Jev was the most accurate of the models we tried at every length. However, its confidence fell more slowly than its accuracy, illustrating that Jev grows more overconfident on long inputs.

The demo below uses Jev's own answers. On the rare-fact questions from [Part 3](#part-3-built-on), its confidence matches how often it's right almost exactly. Split by task, it doesn't: it's overconfident on theory of mind, prompt injection and sorting, and underconfident on grid navigation. Move the temperature and watch: no single setting lines them all up.

![Calibration demo: Jev's confidence against how often it is right, overall and by task, with a temperature slider](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/calibration-demo.png)

### Calibration Needs the Right Labels

Temperature has a limit, though: it softens or sharpens every answer by the same amount. It can't make a model sure on one question and spread out on another, so that has to come from the training labels. If every label is a single best answer, the model learns to be sure even when the honest answer is a spread. Jev 1.13 did exactly this: asked which color comes out of a bag of 50 red, 30 green and 20 blue marbles, it said red at 0.99. Jev 1.14 gives roughly the real odds. Where the right answer is a probability, train on the probability.

## Part 2: Serving

One of the things Jev is known for is speed and cost. Part of this is explained by no decoding - every request gets one output token; however, that's only one part. The other aspect is how Decision Model requests are structured - one state, multiple questions per state, and one response containing all the answers. Real requests ask many questions about the same state, and Jev's latency barely moves as you add more. To match it, a decision model's latency has to stay nearly flat too.

### The Naive Approach

The natural first version on [vLLM](https://github.com/vllm-project/vllm) sends each question as its own sequence: state + question 1, state + question 2, etc. Turn on prefix caching (reusing the already-computed state across sequences) and the GPU only computes the state once.

Unfortunately, latency still climbs with the number of questions, for two reasons:

1. **Cache hits aren't free:** Prefix caching saves the GPU from recomputing the state, but the server still handles the full state for every question. For each one, it parses, hashes and looks up every block of the state in the cache, and the GPU waits on it. That work grows with state length x question count. A request with 64 questions about a 30,000-token state means handling about 1.9 million tokens, about 57x the roughly 34,000 the GPU actually computes.
2. **Queuing:** The scheduler only admits a limited number of sequences and tokens per step. A request with 64 questions is 64 sequences competing for those slots with everyone else's traffic.

The demo below counts the tokens the server handles. The left side is the naive approach, the right side is the fix (more on that next). Move the sliders to see how fast the gap grows.

![Server work demo: one sequence per question repeats the 8,000-token state 32 times, while tree batching handles it once](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/server-work-demo.png)

### Tree Batching

The fix: put all the questions into one sequence.

1. **Lay out the state once:** Then pack every question after it as a branch of a tree. Questions that start the same way share those tokens too.
2. **Mask the attention:** Each question token can "see" the state and its own question, but not its siblings. Each branch also gets the position ids it would've had on its own, so every question gets the same answer it would get in its own sequence.
3. **Run one forward pass:** Read every question's readout token from that single pass.

The demo below shows one packed sequence with a 4-token state and three 2-token questions. Hover or tap a row to see what that token can "see".

![Attention mask demo: each question attends to the shared state and its own tokens, never to a sibling question](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/attention-mask-demo.png)

Now the server handles the state once, however many questions there are, and the scheduler only sees one sequence. **Latency only grows with the question tokens, which are short.**

The chart below plots a simple cost model (illustrative, not a benchmark). Turn on **Busy server** to add the wait for scheduler slots.

![Latency demo: illustrative latency climbs steeply with question count for one sequence per question and stays nearly flat with tree batching](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/latency-demo.png)

Bonus: cache the state across requests too. Systems often come back with a second round of questions about the same document. With the state cached and the tree packed after it, the second round only pays for its questions.

### A Hint in Jev's Limits

Jev's [model docs](https://docs.typesafe.ai/models) list its context limits as "64k per-request / 32k per-question". The per-question limit counts the state plus any one question. The per-request limit counts the state plus all the questions, **with the state counted once**.

That lines up neatly with a tree. The 32k limit caps one path (the state plus one question, which is everything a single question can "see"). The 64k limit caps the whole tree (the state once, plus every branch). If each question ran as its own sequence, the state would count once per question, and a 64k request limit could barely fit two questions about a 30k document. Archer Hume's [black-box study](https://archerhume.com/posts/jevs-architecture-unmasked/) points the same way: Jev's behaviour is consistent with computing the state once and batching the questions.

Move the sliders below to see which limit a request hits first, and how far over it would go if the state counted once per question.

![Jev's limits demo: a request's per-question and per-request token counts against the 32k and 64k limits](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/jev-limits-demo.png)

TypeSafe also offers an opt-in long-context mode, turned on with a request header, that raises both limits to 262k tokens. Its docs warn that it's slower and less accurate.

### Other Serving Details

- **No decoding:** A decision is read from the logits of one forward pass. There's no generation loop and no output tokens to wait for (no "tokens per second" to worry about outside of input tokens/prefill). Through OpenRouter, Jev answers in about 0.2 s at 1k tokens and 0.66 s at 64k, growing almost linearly with input.
- **Single-question fast path:** A tree with one branch is just a regular sequence. Send it through the engine's normal attention and skip the tree machinery.

## Part 3: What Is Jev Built On?

[Part 1](#part-1-training) started from an ordinary LLM. TypeSafe says Jev is not an LLM and not autoregressive, but is transformer-based ([The Batch](https://www.deeplearning.ai/the-batch/models-built-to-do-one-thing-well)) and hasn't said what it started from. Jev's API is public, though, so we ran experiments against it to try to glean more information on it. Every Jev result below comes from Jev 1.13. TypeSafe has since released Jev 1.14, which changed how Jev spreads probability on chance questions (see [Calibration Needs the Right Labels](#right-labels) in Part 1); we haven't rerun these tests on it.

We follow five clues: what Jev says about itself, how it counts tokens, how it answers sensitive questions, what it knows, and how well it answers without thinking. The results painted an interesting picture.

### Asking It Directly

A decision model can't refuse a question. It has to spread probability across the options it's given, so you can ask it things a chat model would dodge. We asked Jev about itself across 14 model families (Qwen, DeepSeek, Llama, Gemma, ChatGPT, Claude, Jev itself, and a made-up "Zorblax" as a floor), using yes/no, two-option, score and pick-one questions, with five different framings of the request. Then we asked the same questions about an unnamed assistant.

Asked about itself, Jev puts Qwen (Alibaba's open model family) first in almost every phrasing. Averaged over all of them, Qwen gets 0.71, with the runner-up getting 0.40. Jev seemingly wasn't taught its own name, which gets only 0.12. When it has to pick one name for an unnamed assistant, it picks ChatGPT, the usual default for a model trained on web text.

![Identity demo: Jev's answers when asked which model family it is, with Qwen on top when asked about itself](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/identity-demo.png)

Decision models don't generate text, so whatever it says here was learned earlier. That puts Qwen somewhere in Jev's history. The next questions are whether Jev **is** a Qwen model, and if so, **which one**.

### Counting Tokens

Jev never shows its tokens, but every response reports how many input tokens it used. Send a request with and without a word, and the difference is that word's token count. [Archer Hume](https://archerhume.com/posts/jevs-architecture-unmasked/) compared Jev's counts against 192 public tokenizers: none matched, and the closest was Qwen's. So we mapped Jev's vocabulary one token at a time, checking whether Jev counts each candidate as a single token.

Jev's vocabulary turns out to be Qwen 3.5's, cut down and topped up from GPT-4o's:

- **Kept:** Nearly all the tokens Qwen 3.5 shares with GPT-4o.
- **Dropped:** Every token only Qwen 3.5 has (many are Chinese, which is why Jev needs more tokens for Chinese than Qwen does).
- **Added:** About 10,000 GPT-4o tokens, almost all of them two kept tokens joined together.

That leaves a vocabulary about 40% smaller than Qwen 3.5's. Rebuilt this way, with GPT-4o's rules for which pieces join first, our copy of Jev's tokenizer matches Jev's count on nearly every one of our measurements, including ones we held back to test it.

![Token demo: Jev's token count for a word beside Qwen 3.5's and GPT-4o's token pieces, with each piece marked kept, dropped or added](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/tokenizer-demo.png)

This is the pattern of a **vocabulary swap on a trained model**. Every token has a learned embedding (the vector the model reads for it). A token both vocabularies share keeps its embedding, so it's free to keep. A new token made of two kept pieces can start from its pieces' embeddings. A token with no kept pieces would have to be learned from scratch, and Jev has almost none of those. A model trained from scratch would have no reason to follow Qwen 3.5's vocabulary this closely, so Jev is likely a Qwen 3.5-family model (Qwen 3.5 and its later releases share one vocabulary) with its vocabulary swapped. Part 4 looks at how that's likely done.

### Sensitive Questions

Qwen models are known to follow the Chinese government's line on topics like Tiananmen Square and Xinjiang. If Jev is built on Qwen, some of that might have come along. We asked Jev 60 questions with widely documented answers: sensitive China topics, matched events from other countries (Kent State, the Gulag, the internment of Japanese Americans) and plain facts about China, in English and Chinese. We asked Qwen 3.5, Llama 3.3 and Gemma 3 the same questions, reading the probability of their first answer letter. On a few Tiananmen questions Qwen mostly starts to refuse instead ("I..."), so there we read the letters from what's left.

![Stance demo: the probability of the documented answer on sensitive China questions for Jev, Qwen3.5-35B-A3B, Llama 3.3 and Gemma 3](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/stance-demo.png)

Llama and Gemma give the documented answer almost every time. Jev and Qwen don't, and they stray on the same questions. Question by question, how far Jev strays lines up closely with Qwen's, while Llama and Gemma barely stray at all. The clearest case is Xinjiang: asked what the facilities holding Uyghurs are, both Jev and Qwen put most of their weight on "voluntary vocational training schools" or "they do not exist" rather than "detention camps". All four models call the camps for Japanese Americans, the Gulag and North Korea's prison camps what they were. The lean is stronger when Jev is asked in Chinese, though its Chinese answers are shakier overall. Most other Qwen models line up about as well, so this points to Qwen but can't say which one.

### Which Qwen?

The Qwen 3.5 family comes in sizes from under 1B to 397B-A17B. In a name like 35B-A3B, the model has 35B parameters but uses only about 3B of them for each token (a mixture-of-experts model). Two things survive fine-tuning and can tell these apart:

- **Rare facts:** Fine-tuning can teach a model new skills, but it rarely teaches obscure facts. Whatever rare facts Jev knows, its base must know too, and a bigger base would know more.
- **Mistakes:** When a fine-tuned model and its base both get a question wrong, they tend to pick the same wrong answer.

We built a quiz from [PopQA](https://huggingface.co/datasets/akariasai/PopQA), a set of questions about people, places and works at every level of fame ("What is [person]'s occupation?"). Each question got 8 options, where the wrong ones are real answers to the same kind of question. Jev, five Qwen models and five open models from other families answered with the options in order and reversed. Every open model answered with reasoning off, except gpt-oss, which can't turn it off and used its lowest setting.

![Knowledge demo: accuracy by how famous the subject is, for Jev and a chosen Qwen model or a model from another family](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/knowledge-demo.png)

Jev's knowledge tracks Qwen3.5-35B-A3B almost exactly, at every level of fame. Qwen3.6-35B-A3B, a later release of the same size, is just as close. The bigger Qwen 3.5 models know far more, and the 27B a little less. Knowledge alone can't single out Qwen, though: Mistral Small 3.2, a 24B model from another family, knows about as much as Jev too.

The mistakes agree. When both get a question wrong, Jev picks the same wrong answer as Qwen's 35B-A3B models nearly as often as two releases of the same Qwen model agree with each other, and far more often than any model from another family, Mistral Small included. Where Qwen 3.5 and 3.6 disagree, Jev sides with 3.5 a little more often. 3.6 looks like an update of 3.5 at the same size, so the two are too alike to tell apart with confidence.

### Answering Without Thinking

Jev answers in one forward pass. It doesn't think first: in our tests, a batch of hard math questions came back as fast as a batch of trivial ones. Yet on many hard questions it answers instantly far better than Qwen3.5-35B-A3B does:

- **[MMLU-Pro](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro):** On hard exam questions, Jev beats it by a wide margin, and even the much bigger [Qwen3.5-397B-A17B](https://huggingface.co/Qwen/Qwen3.5-397B-A17B), both answering instantly.
- **Fresh math:** On calculation questions written by code the same day (so no model could have trained on them), Jev gets most right and Qwen only about half.
- **Unit conversion:** Jev got every one right, Qwen fewer than half.

![Answering demo: accuracy on hard questions for Jev and Qwen models answering instantly](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/answering-demo.png)

That's close to what [Qwen's model card](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) lists for the same model when it thinks first. So on MMLU-Pro, Jev comes close to Qwen's thinking score, without the thinking. It isn't better everywhere, though: on date arithmetic it's barely above chance, no better than Qwen. Two training recipes could do that: label questions with a model that thinks and train the decision model to give those answers in one pass (distillation), or label them with code that checks the answer. Jev's answers on hard questions look like a strong thinking model's, and only slightly closer to Qwen's than to OpenAI's [gpt-oss](https://huggingface.co/openai/gpt-oss-120b). Where those two disagree, Jev usually picks whichever is right, so it doesn't look like a straight copy of Qwen's thinking. Either way, it fits the rare facts: training can teach a model to jump to an answer it could work out, but not facts it never saw.

TypeSafe's founder, Diogo Almeida, has hinted at much the same on the [Latent Space podcast](https://www.latent.space/p/jev). All their data is synthetic ("Yep"), "the goal is, programs in the loop", and on RL with checkable answers (the method behind reasoning models), "zero is the optimal amount for our shape". Programs can check exactly the kind of questions Jev is best at, like math and unit conversion.

### Our Best Guess

Put together:

- **Identity:** Jev calls itself Qwen.
- **Tokenizer:** Jev's vocabulary is Qwen 3.5's after a swap, the way you'd change a trained model's vocabulary.
- **Sensitive questions:** Jev leans the way Qwen does, question by question.
- **Knowledge:** Jev knows what Qwen's 35B-A3B models know, and makes the same mistakes.
- **Context:** Jev's long-context mode tops out at exactly Qwen 3.5's context length, though other recent Qwen models share it.
- **Price:** By our rough estimate (rented B200 GPUs at today's prices), Jev's price covers a 35B-A3B model with room to spare, while Qwen3.5-397B-A17B would barely break even at best.

Our best guess: Jev is likely **[Qwen3.5-35B-A3B](https://huggingface.co/Qwen/Qwen3.5-35B-A3B) or [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B)**, with a swapped vocabulary, trained to answer decision questions in one pass. That also fits TypeSafe's description: a transformer that isn't used as an LLM and doesn't generate text. And it fits how Diogo talks about building on existing models: "If you gave me a billion dollars, I wouldn't pre-train... Frankensteining is not the most elegant, beautiful thing, but it solves problems."

Some caveats: a model can't see inside itself, so the identity and stance results are habits rather than facts. The open models ran through third-party providers, which may run them slightly differently from the originals. And we tested Jev 1.13 only.

It's also a reminder for anyone building their own decision model: fine-tuning mostly changes what a model does, not what it knows or believes. Jev still answers questions about Xinjiang the way Qwen does. Pick a base whose knowledge and beliefs you're happy to ship, and check them before you train.

## Part 4: How They Likely Built It

If Jev is a Qwen model with a new vocabulary, trained to answer in one pass, two questions follow: how do you swap a trained model's vocabulary, and what is the training method TypeSafe calls RLCD?

### Swapping the Vocabulary

So how do you give a trained model a new vocabulary? Each token has two rows of numbers in the model: one it reads when the token comes in (the embedding), and one it uses to score the token as an answer (the output head). Jev's likely base keeps these as two separate tables, with one row per token in each. Given what Jev's vocabulary looks like, the likely recipe is:

1. **Keep:** Copy both rows for every token Qwen 3.5 shares with GPT-4o.
2. **Drop:** Delete the rows of tokens only Qwen 3.5 has.
3. **Add:** Start each new GPT-4o token at the average of its two pieces' rows. This is the standard trick from [Fast Vocabulary Transfer](https://arxiv.org/abs/2402.09977), and tools like mergekit's [tokensurgeon](https://github.com/arcee-ai/mergekit/blob/main/docs/tokensurgeon.md) offer it too. It also explains why almost every added token is two kept tokens joined: averaging needs pieces that already exist.
4. **Train:** Train the model so the new rows settle in, typically the embedding tables first, then everything.

![Swap demo: where Jev's tokens come from, and the keep, drop, add and train steps on toy embedding rows](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/swap-demo.png)

Step 4 can't be seen from outside, and other ways of filling in new rows (like fitting each one from several existing tokens, as tokensurgeon does by default) would leave a similar vocabulary behind.

The tokenizer around the new vocabulary is spliced from both models' published files. From Qwen 3.5 it keeps the text clean-up and the rule for splitting text into words, so every digit is still its own token, the way the model learned. From gpt-oss it takes the order in which pieces are joined, and a "look up whole words first" setting. Many join rules didn't survive the swap: most of Jev's tokens can only appear as whole words, never built from pieces. The rules that did survive mostly form chains, where a token can only be built if the token one byte shorter can. None of the standard conversion methods we tried produce this, so it's likely TypeSafe's own script.

What does the swap cost? On English, almost nothing: Jev uses about as many tokens as Qwen 3.5. Other languages written in the Latin alphabet cost a little more. Russian, Greek, Arabic and Japanese take about twice as many tokens, Chinese a bit less, and Thai almost four times, because both the dropped tokens and the missing join rules leave words in small pieces. Jev bills per input token, so that text costs more to send. Switch the Token Counts demo above to Languages to see each one.

Why swap at all? We don't know, but it isn't to save tokens. One reason stands out: GPT-4o's vocabulary is also the one OpenAI's open gpt-oss models use, and training one model on another's token-by-token probabilities (distillation) needs a shared vocabulary. That would fit Jev's answers on hard questions being nearly as close to gpt-oss's as to Qwen's (see Answering Without Thinking). But Jev is missing about a quarter of GPT-4o's tokens, so the match is partial, and plainer reasons (smaller tables, reusing an existing data pipeline) fit too.

### What Is RLCD?

TypeSafe calls its training method RLCD (Reinforcement Learning for Calibrated Decisions). There's no paper ("No, not yet"), so here's our read from what Diogo has said and what Jev does.

He compares it to RLHF (reinforcement learning from human feedback, the step that turns a pre-trained model into a chat assistant, usually with an algorithm called PPO): "It's not about the PPO. That part doesn't matter... And for us, RLCD is this new task." In other words, RLCD names a goal, not an algorithm: decisions whose probabilities mean what they say, learned from synthetic tasks that programs can check.

To us that reads as **ordinary training on the right labels**. When you know the right answer (or the odds), scoring the model's probabilities against it with cross-entropy is the whole job, and calling it reinforcement learning is mostly framing. Jev's behavior fits:

- **Labels:** Jev 1.13 answered chance questions as if it were sure (the marble bag in Part 1), and 1.14 gets much closer to the odds, though not all the way (see the demo below). That's new training data, not a new method. The fix was narrow, too: asked for a user's favorite color, 1.14 still picks blue with near certainty.
- **Types:** Yes/no answers follow stated odds closely (tell it there's a 50% chance and it says about half) and almost never reach exactly 0 or 1. Choice answers often do, and the same question asked both ways can get quite different answers, so each type seems to be trained on its own.
- **Options:** Choice is a softmax across the options. List the same option twice and the two copies split its probability.
- **Questions:** Each question is answered on its own. The same question gets the same answer alone or among many others.

![Chance demo: Jev 1.13 and 1.14 probabilities on chance questions beside the real odds](https://oumi.ai/blog/how-to-build-a-decision-model-like-jev/chance-demo.png)

So for your own decision model, you don't need a new RL algorithm. Plain cross-entropy gets you there, as long as the labels are probabilities wherever the truth is uncertain, and you check calibration per question type and per task, not only overall.

## What To Take Away

- **Training:** A decision model is an ordinary LLM that answers at one readout position, by scoring reserved option tokens instead of writing text.
- **Serving:** Read each document once and answer all of its questions in one pass (tree batching), so extra questions barely add time.
- **Calibration:** A temperature fixes steady over- or underconfidence, but the labels decide the rest. Train on probabilities where the truth is uncertain, and check each task, not just the average.
- **Base model:** Fine-tuning mostly changes what a model does, not what it knows or believes. Jev still answers like Qwen, so choose your base with care.

## Build Your Own Decision Model

Jev is a general model that every customer shares. But many teams make the same decision thousands of times a day (route this ticket, flag this payment, grade this answer), with their own options, their own labelled history and their own idea of what counts as right. A small model trained on that exact decision and served this way can be faster, cheaper and more accurate on it, and it can run in your own cloud.

That's where we come in. We help teams with:

- **Data:** Turning their labels, logs and rules into a training set of Choice, Yes/No and Score questions.
- **Training:** Fine-tuning and calibrating an open model on it.
- **Serving:** Running it with one-pass, tree-batched inference, in their own cloud.

Oumi's platform can train and serve a decision model like this on your own data. [Book a demo](https://oumi.ai/schedule-demo) and we'll help you build one for your workflow!