When Jev Breaks: Lessons From Building a Fast Web Agent

Jev by TypeSafe AI kept appearing in my X feed, alongside OpenAI’s Decisions API. I wanted to see what happened when we used Jev to make decisions inside a web agent.

TypeSafe AI introduced Jev, a model that makes decisions instead of writing text. Within days, people were wiring it into browser agents and posting runs that finished in seconds. We wanted to know how much of that holds up, so we rebuilt the decision loop of a web agent around Jev and pointed it at real sites.

Jev made decisions quickly enough that the browser, rather than the decision model, took most of each step. It also exposed failures in planning, model handoffs, and page timing. The Wikipedia search below shows how a correct field choice led to the wrong query.

How Jev makes decisions inside a web agent

A conventional language model predicts the next token in a sequence. Its chat responses and tool calls are built from those predictions. Everything it does, from chat to tool calls, is built on that one move.

Jev is a decision model. You hand it some state and a set of typed questions, and it returns answers with probabilities. Take a support ticket: is this customer angry, which team should handle it, how urgent is it? Jev answers all three in one request and generates no text on the way.

A browser agent is an obvious place to try it. Every step is a multiple-choice question: of the things on this page, which one do I act on next? Other teams saw the same thing within the week. Stagehand reported median Act latency falling from 1.97s to 0.46s with Jev as the decision layer. Browser Use posted a flight search that finished in about 7 seconds.

How the Jev web agent decision loop works

The agent runs a four-beat loop: read the page, decide, act, observe. Reading turns the live page into a numbered list of things you can interact with: links, buttons, text inputs, selects. That list becomes the menu.

Deciding is a single Jev request. It carries the goal, a log of what the agent has done so far, and the menu. It asks one multiple-choice question and four yes/no questions:

• Which element should the agent act on next?
• Is the goal done?
• Is the current part of the goal done?
• Is an overlay blocking the page?
• Is there something on screen worth capturing?

The five questions go to Jev in one request. If Jev selects a text field, the agent hands the writing task to a second model. Inside the loop, only the highlighted path involves a second model. That handoff is where the typing failures below came from.

Jev never generates text. Clicks, selects and scrolls go straight to the browser. Typing is the exception: Jev picks the field, and a small text LLM writes what goes in it. A third model writes the final answer from the pages the agent captured.

Around Jev sits plain code: rules that narrow the menu, override a pick, or stop the run. Jev returns probabilities, so those rules can be thresholds. The agent stops when "goal done" passes 0.85, not when a model writes "I think I'm finished".

For a closer look at Jev's decision types and model limits, read What Is Jev AI? TypeSafe's System One Model Explained.

Jev's speed shifts the web agent bottleneck to the browser

A Jev decision takes about a tenth of a second. Across 142 runs and 1,395 decisions, the average was 102 ms, on roughly 8,700 input tokens of page state. For scale, Stagehand's LLM baseline for a comparable step was 1.97s.

That changes where the time goes. Jev accounted for 9% of step time. Acting on the page and waiting for it to settle took 64%. An average step took 1.25 seconds end to end, and most of that was the browser.

The second surprise was quality. In our tests, Jev's picks were about as good as a Sonnet-class model's when both used the same menu and action log. The harness is doing real work in that sentence: Jev can only choose among what you show it, and it remembers nothing between steps, so the menu and the action log decide most outcomes.

Where the Jev-powered web agent failed

Four things broke, and only the first is a limit you can read off a spec sheet.

32K tokens is not much web page

Jev's context window is about 32K tokens. Our average decision already used 8,700, and that is after reducing the page to a list of interactive elements. Raw HTML is out of the question. Everything in the request has to earn its place: the goal, the action log, the menu.

No persistent plan hurts long tasks

Jev answers the question in front of it. It does not think a plan through, and it carries nothing between steps. That is fine for "open the pricing page". It is not fine for this:

Find the price of a FedEx Medium Box at FedEx Priority Overnight in the Local, Regional and National zone tabs.

That goal has three parts, and the agent has to know which are finished. Left to infer that from its own log, it loses track on long runs.

We added a step before the loop that splits a goal into sub-goals. One Jev call decides whether the goal has several parts, and a small LLM writes them out. Every decision then sees the list with the current part marked. This helped. It is not a full solution, because the plan is fixed before the agent has seen the site.

Mixing models opens a new kind of failure

Typing is where it broke most visibly. Jev picks the field and a small LLM writes the text. We asked:

What are the first lines on the Wikipedia pages for basketball and quantum physics?

The right move is to search "basketball", record the line, then search "quantum physics". Jev picked the search box, correctly. The text model, handed the whole goal and one field, typed "basketball and quantum physics". Wikipedia answered that the page does not exist.

Neither model was wrong on its own terms. A decision model returns a choice and nothing else. It cannot write a note telling the next model why it picked that field. Whatever the text model needs has to be carried around Jev by the harness, and anything you forget to carry turns into confident, wrong text.

The browser becomes the hard part

Once a decision takes 100 ms, everything else shows. Waiting for a page to settle is now a large share of every step, and waiting too little means Jev decides on a half-rendered page. An agent that acts about once a second without pause also looks less like a person, so bot avoidance matters far more than it did when a slow model was pacing the run.

Lessons for building a fast web agent with Jev

Treat Jev as the deciding component of an agent, not as the agent. In these runs, it answered multiple-choice and yes-or-no questions quickly. Planning, writing and memory have to come from somewhere else.

• Spend your effort on the menu. Jev can only choose among what you show it, and you have 32K tokens to show it in.
• Use the probabilities. On one location picker, "nothing here dismisses this overlay" won at 0.32 against a close button at 0.30, and the agent clicked around under the overlay. We now require "none" to clear 0.5 before believing it.
• Carry context across every handoff yourself. Jev cannot brief the next model, so the harness has to.
• Budget for the browser. Measure settle waits separately from decision time, and check whether faster actions trigger bot defenses on the sites you use.

P.s. If you think this is super cool and absolutely needs to be in some kinda web agent. Stay tuned to the work we're doing at tinyfish. The Jev moment for web agents is not too far away.

Want to test a web agent on a real site? Try the TinyFish Playground. The Playground runs TinyAgent, not the experimental Jev-powered loop described above.

Keep reading

• Building a Browser for the Agent Era explores the browser infrastructure behind agent execution and page-settle detection.
• Headless Browser vs AI Agents for Web Automation: How to Choose explains where browser control ends and agent decision-making begins.