Decido
A multimodal web agent for studying why multi-step browser agents fail—and how much of the answer lies around the model rather than inside it.
Where it started
Decido grew from two related ideas. While working as a research assistant at the UW Computer Systems Lab, I was studying whether personal agents fail because they need more training or because they lack context about their own deployment: what they have already done, what comes next, and whether the last action actually worked.
I also wanted to build a web agent after reading AI2's MolmoWeb paper. Decido borrows its idea of planning with named web skills instead of raw clicks, but takes a different route: it pairs a planner that reads the DOM with one that reads screenshots, then ranks their proposed plans against each other.
How it works
For each task, a DOM planner and a vision planner propose high-level plans such as filling a form, applying filters, or adding an item to a cart. A ranking layer looks at their confidence, whether the two sources agree, the order of the remaining checklist items, and whether a plan repeats work that has already succeeded.
The winning plan is compiled into primitive browser actions without another model call. Playwright executes and verifies each primitive, and only verified evidence advances the task checklist. An independent evaluator checks any final success claim before the session is allowed to end.
The experiment I cared about
I ran 34 tasks across six categories three times per configuration. After applying the same shared bug fixes, the older atomic runner completed 70 of 102 sessions. Moving to the skill runner with only the DOM planner completed 93 of 102. Adding the vision planner reached 99 of 102.
The models stayed the same. Giving them a checklist, ordering context, memory of completed work, and verified execution feedback produced the largest gain.
The comparison is the concrete version of the lab hypothesis: the same underlying intelligence can perform very differently depending on what the surrounding system tells it about the task and its own prior actions.
Why verification mattered
Browser agents are generous graders of their own work. A page containing the words “Add to cart” does not prove that anything was added, and a confirmation element can exist before a form has been submitted. Most tasks therefore include independent URL or text checks against the final page, and those checks override the agent's self-report in either direction.
Seven of 60 apparent successes from the atomic configuration failed that outside verification. In the skill runner, one of 102 did. The gap made verification part of the result rather than a final testing detail.
What I learned
Vision helped most when the page itself was visually ambiguous: comparing many prices at once or finding a bare, unlabeled input. It was not universally better, and the DOM-only configuration even won one more run on a multi-page subset. A second planner can add noise as well as signal.
The benchmark also has deliberate limits. It uses live demo sites rather than WebArena-scale tasks, the ranking weights are still hand-tuned, and vision proposes plans but does not ground individual clicks. I see the results as evidence about failure modes and deployment context, not a state-of-the-art claim.