First tests with Jev and other decision models

Published AI Braindump

I spotted Jev on X a few weeks ago, right after TypeSafe launched it. Since then I've tested it in a few client applications and compared it against other models.

Jev is a decision model. You use it for choices with fixed answers, such as deciding whether an email is urgent. A large language model (LLM) can do that too, but it writes out its answer token by token, and even with a fixed output format your code still has to parse the response. For a simple decision, that's quite a detour.

What a decision model does

You give it some context and a few questions with fixed options. For each question you get back a choice, a score or a yes or no, each with a probability between 0 and 1. No free-form text comes out. TypeSafe calls them "smart if-statements".

With that probability, you pick a threshold yourself: above it the model decides automatically, below it a person takes a look.

According to TypeSafe, Jev answers all your questions in one go, in 70 to 500 milliseconds. On its site it compares a workflow of 0.11 seconds with Jev against 8.6 seconds with an LLM, though those are its own measurements.

Other decision models have arrived since. Cloudflare released Clef on 1 October, open-weight under Apache 2.0 and with the same application programming interface (API) as Jev. OpenAI has its Decisions API in public beta, which it says is about ten times faster than its regular Responses API.

My own tests: RAG and 150 emails

In my first test, I put a decision model in front of retrieval-augmented generation (RAG). Instead of routing every question through an LLM first, the decision model checks straight away whether a lookup is needed and fires the RAG query directly.

I also ran a test on a batch of about 150 emails, categorising each message and archiving anything irrelevant. I compared Jev, OpenAI's decision model and a standard flow with Claude and Codex.

Jev and OpenAI reached similar accuracy, but Jev was faster and cheaper than OpenAI. The advantage of OpenAI's decision model is that you don't have to maintain an extra integration if you're already using its platform. My results with Jev line up with a larger test by MindStudio on 1,000 emails, where Jev answered seven questions per email in parallel in about six seconds, compared with five minutes for a single category using a GPT-5.6-class model.

When I still bring in an LLM

If that decision needs to be followed by a written reply to a customer, I still bring in an LLM to turn the technical details into a clear answer.

So in practice I combine them: an email or question comes in, and the decision model immediately decides whether RAG is needed, how urgent the message is or whether it can be archived. The LLM only steps in when a reply actually needs to be written.