Anthropic released Claude Haiku 5.5 on October 7 and pitched it straight at browser work. The launch post calls it a good fit for “speed-sensitive tasks like live customer support and browser use”, the product page lists form filling, data entry, and moving information between apps, and the same release added computer use and browser use to Anthropic’s Python and TypeScript SDKs in beta. On OSWorld 2.1, the computer-use benchmark Anthropic reports, Haiku 5.5 scores 72.4% for about $0.61 per attempt at max effort, and 53.3% for about $0.13 at the default medium effort.
The headline number is a partial-credit score. Read the system card and the same runs have a strict pass rate of 37.1%. So a team that wires Haiku 5.5 into a browser lane gets a run that costs cents and a result that, more often than not, is partly done. The run got cheap. Checking it did not, and for a team running several agents that moves the real cost of browser automation onto whoever reads the results.
TL;DR
On Anthropic’s OSWorld 2.1 offline subset, Haiku 5.5 averages 72.4% partial credit but fully passes 37.1% of attempts; Sonnet 5.5 and Opus 5.5 show the same 34 to 35 point gap at higher prices. Treat every browser run as partly done until checked: write a strict acceptance check into the Task, make the Agent stop before any submit action and report done, not done, and evidence, and let a person release the final step with a comment that @ mentions the Agent. Spend effort to save review time; the run is the cheap part. Agents research, execute, test, and report. People set direction, grant authority, and accept the result.
Partial credit is not done
Anthropic ran OSWorld 2.1 on 82 of its 108 tasks at 1080p, with up to 500 steps per attempt, at max effort, averaged over five trials. It reported two numbers per model, and the second one is the one a reviewer lives with.
| Model (max effort) | Partial credit | Strict pass | Gap | Cost per attempt | Cost per strict pass (our arithmetic) |
|---|---|---|---|---|---|
| Haiku 5.5 | 72.4% | 37.1% | 35.3 pts | $0.61 | about $1.65 |
| Sonnet 5.5 | 83.9% | 48.8% | 35.1 pts | $5.73 | about $11.75 |
| Opus 5.5 | 87.2% | 53.2% | 34.0 pts | not charted | not charted |
First, the gap is not a Haiku problem. Every model in the family leaves about 35 points between “most of the steps happened” and “the task is done”, and even Opus 5.5 fully passes only about half the time. A large share of computer-use attempts, as measured here, end partly finished. Second, Haiku 5.5 makes partly finished work very cheap to produce. Per strict pass it costs about a seventh of Sonnet 5.5, and per attempt about a ninth.
Partly finished is the dangerous case for browser work. A run that fails outright leaves nothing behind. A run that moves 31 of 40 records into the CRM, or fills a vendor form up to the tax field, leaves state in someone else’s application that looks finished from a distance. Whoever reviews it has to find the nine missing records, and nothing on the screen points at them.
Where the cost goes when the run costs cents
A cheap browser lane changes which line of the budget you watch. At $0.13 per attempt, a hundred form-filling runs is a rounding error on the model bill. A hundred runs that each need a person to confirm what got entered can be a day of someone’s attention. Haiku 5.5 is priced 90% lower than Haiku 4.5 on prompts up to 100K tokens; the reviewer is not.
There is a price wrinkle too. Haiku 5.5 is $0.10 per million input tokens and $0.50 per million output for prompts up to 100K tokens, and $0.50 and $2.50 for prompts over 100K. A long browser session can grow its prompt past that line, and from then on each request pays the higher rate. That is one more reason to keep browser Tasks short: one form, one batch of records, one Task.
Human review is the bottleneck and agent workdays per human workday made this argument before. What Haiku 5.5 changes is the ratio: at $0.13 per run, the run hardly registers, and cheap models can cost more once the review queue is counted. The only lever left is making each check faster.
Computer use agent review: what the Task has to carry
A browser-agent Task is reviewable when checking it takes less time than redoing it. That depends on three things the Agent cannot invent after the fact: a strict acceptance check written before the run, a report that separates done from not done, and evidence a person can open without replaying the session.
In Sharkly the Task is already the shared record. It holds the description, comments, attachments, and activity; the Agent’s text for the team is stored on the Task as a comment linked to the run; and the execution log records tool calls and Runtime events. So the review design lives in two places: the Task description says what “done” means, and the Agent instructions say what the report must contain. A starting point for the instructions of a browser-entry Agent:
Restate the acceptance check from the Task description before you start.
Work up to, but not including, any submit, send, publish, pay, or delete action.
Then stop and report, in this order:
Done: every record or field you completed, by its identifier.
Not done: everything you skipped or could not complete, and why.
Check: the acceptance check, and whether your own count meets it.
Evidence: the paths of the screenshots you saved in the task directory.
Leave the Task waiting for human review. Do not take the final action
until a person @ mentions you in a comment on this Task to release it.
Write the acceptance check as something countable: “40 rows in the CRM, each with a matching email”, not “import the spreadsheet”. The “Not done” list is the part partial credit hides; naming what was skipped turns a 72% run into a 72% run with a known remainder. Our guide to reviewing agent work with task-level evidence goes further on report formats.
Per Sharkly’s Agents documentation, the Agent follows the Runtime’s default model; to change it, specify one in Custom arguments on the Agent or Runtime (claude-haiku-5-5 for Claude Code). There is no model dropdown. Setup is in how to run Claude Haiku 5.5 agents in Sharkly.
Stop before submit
Partial is acceptable for a draft and unacceptable for a submitted form. So split the irreversible step out of the run instead of hoping the model gets it right.
The pattern uses parts Sharkly already has. The Agent works up to the final action and leaves the Task waiting for human review, which surfaces in the reviewer’s Inbox. The reviewer opens the report and the screenshots, fixes or rejects, and then @ mentions the Agent in a comment on the Task to release it. A plain comment does not start the Agent; the mention opens or continues its conversation on the Task, and its reply is stored as another comment linked to the run. The submit happens in the same Task, under the same record, after a person said yes. Our approval gates guide covers where else to put that stop.
The gate also limits the other browser risk. A browser agent reads pages it did not write. Anthropic’s system card says Haiku 5.5 resists prompt injection better than any earlier Haiku, largely matching its frontier models against adaptive attackers in computer-use environments. That is a benchmark claim, not a promise about your vendor portal, and an injected instruction that cannot reach the submit button without a human comment is a much smaller incident. The host-side boundary (a dedicated Computer, credentials bound per Agent through its secret environment variables, one browser run at a time) is the same as in our computer-use guardrails guide, and prompt injection and agent guardrails covers the rest.
A decision pair for the lane. Use a Haiku 5.5 browser Agent when the result can be checked against a count or a list in under a minute, and the final action is reversible or gated behind a comment. Keep the work with a person, or a larger model with the same gate, when checking means redoing the work, because then a 37% strict pass rate buys you nothing but a second pass.
Pay for effort, save on review
Haiku 5.5 is Anthropic’s first Haiku with an effort setting, and the default is medium. On OSWorld, effort moves the partial-credit score a long way for very little money:
| Effort | Partial credit | Cost per attempt |
|---|---|---|
| Low | 42.0% | $0.07 |
| Medium (default) | 53.3% | $0.13 |
| High | 61.3% | $0.18 |
| Xhigh | 67.6% | $0.28 |
| Max | 72.4% | $0.61 |
Going from medium to max adds about $0.49 per attempt and 19 points of partial credit. If those points shorten the review even slightly, max is the cheaper setting for any browser Task a person checks. The system card charts strict pass rate by effort but states the number only at max, so measure the strict outcome on your own Tasks before you pick a lower level. Effort is set in Claude Code on the Computer: the Agent exposes no model picker or thinking-depth control. Run the cheap lane cheap where nobody reads the output, as in Luna for the boring eighty percent; a browser lane is not that.
Browser work that writes into other people’s systems gets its own Task rather than living inside a bigger run, because someone has to accept it; Haiku 5.5 as a subagent or its own Agent covers the general rule.
A cheaper browser agent means more runs, and more runs means more partly finished results waiting for someone to check them. If you’re already running several agents, Sharkly gives you one place to manage their tasks and results.
