What Is Jev, and Can You Run One on Your Own GPU?

📚 More on this topic: Jev Mode on a 3090 · Best Local LLMs for Structured Output · Intent Engineering for Local AI · InsiderLLM Benchmarks

Somewhere in your stack there is an LLM doing a classifier’s job. It sorts tickets, or decides which tool a message goes to. It does this by writing an answer one token at a time — brace, quote, field name, colon, value, and on for twenty tokens so a parser can pull out the one word you wanted. Half a second per decision on a 27B, most of it punctuation.

Jev is Typesafe AI’s model that answers that kind of question by choosing from options you hand it, every field scored in one forward pass, each answer with a probability attached. Typesafe calls it a “System One model,” after the fast, automatic half of the thinking in Kahneman’s book. It is not a chatbot. It does not reason and it does not explain itself. It writes no output tokens at all — which is why the output side of the bill is zero. You give it a situation and a short list of outcomes, and it tells you which one, and how sure it is.

The three primitives, as code

On Latent Space this week, Diogo Almeida, Typesafe’s CEO and a co-author of the InstructGPT paper, put the whole API in terms a programmer already has:

Choice picks one option from a list. His words: choice “maps into a switch statement on an enum.”
Noul answers a yes/no question with a probability. The name comes from Bernoulli, he says, and it is “Bool-ish,” true or false but continuous. “Noulis map to if statements.”
Score rates against ordered levels. “Scores map to sorting or thresholding at a greater than or less than.”

The docs let you mix all three in one call, and every question is evaluated in parallel against the same input. That last part is the mechanism — a written answer has to spell out field two after field one; a chosen answer scores every field from the same prefix at once, so adding fields adds almost nothing to the clock.

Almeida is careful to say these are new types, not types that exist in a language yet. Fair enough. I still find the mental model holds: if you would have written a switch, an if, or a sort on the output of a prompt, that is the shape Jev is built for.

Is your problem Jev-shaped?

Messy text in, a short fixed list of outcomes out, the answers independent of each other — that is the test. If your fields depend on each other, or the right answer needs a chain of steps, you are back to an LLM.

Nate B. Jones’s explainer, “Why Developers Are Losing Their Minds Over AI That Can’t Write”, frames Jev as a general-purpose classifier and sorts the uses into four patterns. One example each:

Pattern	What it looks like	Example
Shim in front of code	A decision gate before a function runs	“Is this message asking for a human?” as a Noul before the escalation path
Sorter over a pile	The same question asked of ten thousand items	Tag every support ticket with a product area and a severity
Chooser in an agent’s outer loop	Which tool, which mode, which skill	Route a user turn to `rag`, `code`, or `chat` before the big model runs
Ambient judgment inside ordinary software	A column that thinks	A spreadsheet column that reads the row’s free-text note and marks it “follow up” or “closed”

The fourth one is the one I keep thinking about. At a few cents per million input tokens and a sub-second answer, a judgment call becomes cheap enough to sit inside software that was never an “AI feature.” It is a function, and it costs about what a function costs.

Price, and the claims

Typesafe’s own models page lists jev-1.13 at $42 per billion input tokens, which is 4.2 cents per million, with no charge for output. Context is 64k per request, 32k of it for the state. Weights are closed and the model is API-only.

The launch page headlines “193.6x faster, 244.6x cheaper” on Typesafe’s own System One tasks, and “zero hallucinations.” Those are the vendor’s numbers on the vendor’s tasks — nobody outside has checked them. The more useful figures come from people using it:

Vercel’s launch post reports that Jev reached “nearly 13% of paid teams” on their AI Gateway within 24 hours, “2x the GPT-5.6 family and more than 6x Fable 5.1’s share.” That is a report of adoption on one gateway, not a market share.
Jones’s video relays two developer reports: Nakshatra Saxena’s, that Jev came in roughly 34x cheaper and 6x faster than a frontier LLM on classifier work, and Zack’s, that it sorted about 20,000 items for a dollar. Those are their figures as relayed in the video, not Jones’s own runs and not mine.
Almeida said on the podcast that Typesafe has passed a trillion tokens a day. His claim, unaudited.

None of that is a measurement I made. Typesafe declines public benchmarks, by Almeida’s own account, in favour of private evaluation, so there is nothing independent to check the 193x against. Treat the multiples as marketing until you have run your own items through it.

You can run the idea today, on your own card

Here is the part that matters for this site. Nothing about this needs Typesafe’s weights. It is a decoding trick — score every option of a schema from one cached prefix instead of generating tokens — and it works on any model you already have.

Harsha Gondala’s open reconstruction put the mechanism on Apple Silicon under Apache 2.0, and reports 5.6x to 7.0x on a 28-field extraction task against ordinary decoding. Codacus took the same idea into a parallel-decision branch of llama.cpp, demonstrated in his video, which adds a /v1/decision endpoint to llama-server. His figure was 300 ms against 3.5 s on Gemma 4 12B.

I ran that branch on my RTX 3090 with Qwen3.6-27B fully on the card, against the 47-item intent split this site has been scoring since August. The full write-up is here; the short version:

Choosing costs about what prompt processing costs. One decision, 21 options across three fields, took 0.298 s — 124 ms to prefill, 169 ms to score every row in one batched pass. Writing the same answer runs on a line of 275 ms plus 26 ms per token.
So the speedup is exactly the tokens you skip. 1.3x when the written answer is four tokens. 5.5x when it is fifty. The 5x line lands at about fifty written tokens, and no amount of schema makes it faster than that.
Same accuracy. Choosing scored 21 of 47, writing 23 of 47 — same model, same mistakes on the same items. Nothing about picking makes the model smarter.
The confidence is informative, not calibrated. Wrong answers averaged 0.62 on their weakest field, right ones 0.80, so a 0.5 gate catches a third of the errors for one lost right answer. But one item came back wrong at tool 1.00. Almeida’s explanation fits: a chat-tuned base is trained to be overconfident, and Jev’s calibration comes from RLCD, a training step the open fork does not have.

That last point is the honest gap between the open path and the product. The mechanism is public and it works. The calibration is the thing Typesafe built, and nobody has reproduced it.

For a local reader the argument is simple. If you already have a 27B on a 3090 doing routing, the branch gives you a parser that cannot fail and a decision in 300 ms, on the model you already trust, with no API key and no bill. What it does not give you — yet — is the probability you can set a threshold on and walk away.

Where it falls over

Fields that depend on each other. Every field is scored from the same prefix and cannot see what the others chose. On my split the field whose right answer depends on the other two came out the same either way, so it was not the cost I expected, but a schema where B is defined by A is the wrong shape for one pass.

Judgment the model does not have. Confident and wrong — that is the failure mode. The type guarantee is that the answer is in your list. It is not that the answer is right, and a 0.98 next to a wrong value is worse than no number at all.

Multi-hop. If the answer needs a lookup, then a comparison, then a decision, that is three questions and some code, not one Jev call. Almeida is clear that it is not a reasoner, and the use-case map is all classification, routing, scoring, and extraction.

On the local path, two hardware caveats from Codacus. Hybrid models with recurrent layers lose the fused batch, so they do not get the speedup. And an offloaded MoE copies experts over PCIe for any batch over about thirty tokens, which is exactly what scoring a schema is; a Flash-Next spilling to system RAM is the case to expect trouble on. Everything I measured was a dense model, fully resident. If your card holds the model, you get the number. If it does not, you probably don’t, and I have not tested the case where it doesn’t.

Limits

Jev’s weights are closed and it is API-only; nothing here reproduces Typesafe’s own numbers, and the vendor publishes no public benchmark to reproduce. The open path is the only one I measured: one model, one card, one 47-item task. Every cost and adoption figure above from a video or launch post is that source’s report, not mine.