dsh-browser-vision — a browser tool for DeepSeek that can see the page
A fully contained browser-use tool driven by DeepSeek through DeepSeek's OpenAI-compatible API. It is designed as a cheap and fast alternative to running browser-use with a general-purpose model, and to be callable from any agent or harness (Codex, Claude Code, OpenCode, custom evals, CI, ...) via a plain CLI with a machine-readable JSON mode.
Since DeepSeek shipped deepseek-v4-flash-vision-exp, the agent can look at the page, not just read its DOM. That model is priced identically to the text Flash model, and a screenshot costs at most 384 input tokens, so vision is no longer a premium feature to be switched off for cost. It is on by default, and the eval suite says that is also the cheapest setting.
What makes it cheap and fast (all overridable via env):
- DeepSeek Flash (
deepseek-v4-flash) / Flash Vision (deepseek-v4-flash-vision-exp) — far cheaper and faster than GPT-class models, and the same price as each other.
- Vision by default (
DSBROWSER_VISION_MODE=on) — a screenshot every step, because a blind agent burns whole step budgets on tasks it cannot see, and steps cost more than pictures.
- Flash mode, no judge, no planning — fewer extra LLM calls per task.
- One shared browser — a single Chrome instance is reused across all tasks in a process; the eval suite runs all cases in one browser instead of one Chrome per case.
Vision
DSBROWSER_VISION_MODE (or --vision) has three settings:
| Mode | Model | Behaviour | Use it when |
|---|
on (default) | deepseek-v4-flash-vision-exp | A screenshot rides along with every step | General use, and everything visual |
auto | deepseek-v4-flash-vision-exp | DOM-driven; browser-use registers a screenshot tool the agent calls when it decides it needs one | Long text-heavy runs where most steps genuinely need no picture |
off | deepseek-v4-flash | DOM only, no screenshots ever | Pure text and form work, or when no vision quota is available |
Vision works the same headless and headed — screenshots come off the same CDP call either way. dsbrowser "task" --no-headless --vision on is a supported way to watch the agent look at a page, and the eval suite takes --no-headless for exactly that.
What vision unlocks, which the DOM cannot answer at all:
- text drawn on a
<canvas> (voucher codes, editors, map labels)
- text baked into an image (scanned invoices, screenshots-in-pages, banners)
- charts and any answer that lives in a shape rather than a string
- "which of these is highlighted / greyed out / overlapping" layout questions
Measured
Six cases, three attempts each, per mode, on the offline fixture suite (dsbrowser-eval --vision all --repeat 3), off-peak pricing:
| mode | passed | DOM cases | vision cases | median latency | screenshots | cost |
|---|
off | 7/18 | 7/9 | 0/9 | 33.7s | 0 | $0.1021 |
auto | 14/18 | 5/9 | 9/9 | 8.3s | 10 | $0.0310 |
on | 18/18 | 9/9 | 9/9 | 6.6s | 111 | $0.0318 |
The surprise is the cost column. on sent 111 screenshots and still cost a third of what off cost sending none. A blind agent does not fail fast, it flails: it spends its whole 20-step budget re-reading a DOM that does not contain the answer, and twenty text steps cost far more than one 384-token picture. Vision is not a premium you pay for accuracy here — it is what makes the run short enough to be cheap.
auto is the weak middle. It solved every vision case, but it did worse on the plain DOM cases than either extreme, and it saves nothing over on.
One machine, one afternoon, n=3. Re-run it yourself before trusting it: dsbrowser-eval --vision all --repeat 5.
Cost of looking
DeepSeek resizes every image so its pixel count is roughly that of an 800×800 image, which caps an image at 384 input tokens. At the off-peak cache-miss rate of $0.22 / 1M input tokens, a screenshot costs about $0.00008. A whole 20-step run with a screenshot at every step adds under a cent.
Two consequences worth knowing:
- Never use
detail: low. It crops to 512×512 first, and on-page text stops being legible — measured on the fixture, the model invented a confirmation code rather than reading it. Every image this tool sends is pinned to detail: high.
- Downscale the capture, not the detail.
DSBROWSER_SCREENSHOT_SIZE=1024x768 shrinks the upload without changing what DeepSeek tokenizes.
Use it from DeepSeek Harness
This repo is also a DSH plugin. Install the plugin, then the engine:
dsh plugin --profile web add "github:tristan-mcinnis/dsh-browser-vision"
uv tool install "git+https://github.com/tristan-mcinnis/dsh-browser-vision"
Restart the profile afterwards (the web profile has HMR disabled). Put DEEPSEEK_API_KEY in the environment — the plugin never stores or forwards a key, it only inherits the environment the engine runs in.
Three tools are registered:
| Tool | What it does |
|---|
browser_vision_task | Run a web task in plain language and get the answer. Navigates, clicks, types, fills forms, and reads what is only visible as pixels. |
browser_vision_extract | Extract fields from a page as JSON validated against a schema you pass in, so nothing has to be re-parsed downstream. |
browser_vision_status | Report whether the engine is installed, so an install problem is distinguishable from a task failure. |
Plugin options (in cordis.patch.yml, or Settings → Plugins):
| Option | Default | Meaning |
|---|
command | dsbrowser | Path to the engine executable |
vision | on | Default vision mode for tool calls |
maxSteps | 20 | Default step cap |
headless | true | Run Chrome headless |
timeoutMs | 300000 | Hard timeout for one task |
Requirements
- Python 3.11 or 3.12
- A Chrome/Chromium binary on the system (browser-use ≥0.13 drives Chrome over CDP — Playwright is not used, so
playwright install is neither needed nor available)
Install
With uv (recommended):
cd ~/Documents/code/deepseek-browser-use
uv venv --python 3.12 .venv
VIRTUAL_ENV=.venv uv pip install -e '.[dev]'
cp .env.example .env # then add your DEEPSEEK_API_KEY
With pip:
python3.12 -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'
This installs the dsbrowser package plus two console scripts: dsbrowser (alias deepseek-browser) and dsbrowser-eval.
Secrets: plain env or 1Password CLI
The DeepSeek key can come from the environment/.env (DEEPSEEK_API_KEY) or from 1Password through the op CLI. Point DEEPSEEK_API_KEY_OP at a secret reference and the tool resolves it with op read --no-newline:
# .env
DEEPSEEK_API_KEY_OP="op://Personal/DeepSeek/api key"
op:// references also work directly as DEEPSEEK_API_KEY. The op CLI must be installed and authenticated (op signin, or export OP_SERVICE_ACCOUNT_TOKEN for headless/CI use). No keychain involvement in the tool itself — and Chromium is explicitly steered away from the macOS keychain too (see below).
CLI
dsbrowser "Open example.com and return its page title"
dsbrowser "Read the voucher code drawn on the canvas at example.com/ticket" --vision on
dsbrowser "task" --vision off # DOM only, cheapest floor
dsbrowser "task" --json # machine-readable output
dsbrowser "task" --max-steps 40 # override step cap
dsbrowser "task" --no-headless --vision on # watch it, and let it look
dsbrowser "task" --model deepseek-v4-pro
python -m dsbrowser "task" # module form
Structured output
Reading a scanned invoice with --schema, which is the whole tool in one call — see the page,
extract the fields, return validated JSON:
{
"invoiceNumber": "5512-B",
"totalDue": "$128.40",
"lineItems": [
{"label": "Standing desk", "amount": "$96.00"},
{"label": "Cable tray", "amount": "$18.40"},
{"label": "Delivery", "amount": "$14.00"}
]
}
Nothing above is in the DOM: the invoice is a PNG. One LLM call, 10.9s, $0.0013.
Pass a JSON Schema and the answer comes back as a validated object instead of prose, so a calling
agent does not need a second model call to parse it:
dsbrowser "Open https://example.com/pricing and list every plan" --json --schema '{
"type": "object",
"properties": {
"plans": {"type": "array", "items": {
"type": "object",
"properties": {"name": {"type": "string"}, "price": {"type": "string"}},
"required": ["name", "price"]
}}
},
"required": ["plans"]
}'
--schema also takes a path to a .json file. Objects, arrays, primitives, enums, unions and local
$refs are supported; the top level must be an object, so wrap a bare list in a property. Optional
properties come back as null rather than forcing the agent to invent a value it could not find.
For prose rather than fields, --markdown asks for the page content as clean Markdown with the
navigation, ads and cookie banners left out.
JSON contract (for agents and harnesses)
With --json, stdout carries exactly one JSON object and stderr stays clean:
{
"tool": "dsbrowser",
"version": "0.2.0",
"success": true,
"result": "PLUM-4482",
"steps": 3,
"duration_s": 14.2,
"error": null,
"model": "deepseek-v4-flash-vision-exp",
"vision_mode": "auto",
"usage": {
"llm_calls": 4,
"images_sent": 1,
"prompt_tokens": 18422,
"completion_tokens": 611,
"cached_tokens": 12800,
"cost_usd": 0.00119
}
}
usage.images_sent is the number of screenshots the model actually looked at — in auto mode that is how you tell whether the agent needed to see the page. cost_usd uses DeepSeek's published rates and accounts for the peak/off-peak window and cache hits; it is an estimate, not a bill.
Exit codes: 0 task completed · 1 task failed (agent/network/model error, error is set) · 2 configuration error (e.g. missing DEEPSEEK_API_KEY, or vision on with a model that has no image input).
Calling it from other agents
Any harness that can run a shell command can use it:
result=$(dsbrowser "Summarize the pricing page at https://example.com/pricing" --json)
# then parse $result with jq: .result, .steps, .usage.cost_usd, .error
# Python harness
import json, subprocess
out = subprocess.run(["dsbrowser", task, "--json"], capture_output=True, text=True, check=False)
payload = json.loads(out.stdout)
print(payload["result"], payload["usage"]["cost_usd"])
Or use it as a library:
import asyncio
from dataclasses import replace
from dsbrowser import Config, DeepSeekBrowserAgent
async def main():
config = replace(Config.from_env(), vision_mode="on", headless=False)
async with DeepSeekBrowserAgent(config=config) as agent:
outcome = await agent.run_task("Read the code on the canvas at example.com/ticket")
print(outcome.result, outcome.images_sent, outcome.cost_usd)
asyncio.run(main())
Configuration
| Variable | Default | Meaning |
|---|
DEEPSEEK_API_KEY | — | required (or DEEPSEEK_API_KEY_OP) |
DEEPSEEK_API_KEY_OP | — | 1Password CLI secret reference (op://Vault/Item/field) resolved via op read |
DEEPSEEK_BASE_URL | https://api.deepseek.com | OpenAI-compatible endpoint |
DEEPSEEK_MODEL | deepseek-v4-flash | model used when vision is off |
DEEPSEEK_VISION_MODEL | deepseek-v4-flash-vision-exp | model used when vision is auto or on |
DSBROWSER_VISION_MODE | auto | off · auto · on (legacy DSBROWSER_USE_VISION=true/false still parses) |
DSBROWSER_SCREENSHOT_SIZE | — | WIDTHxHEIGHT to downscale screenshots before upload |
HEADLESS | true | headless browser (vision works either way) |
MAX_STEPS | 20 | max agent steps per task |
DSBROWSER_CHROME_ARGS | --password-store=basic,--use-mock-keychain | extra Chrome flags; defaults keep Chromium off the macOS keychain |
DSBROWSER_FLASH_MODE | true | browser-use flash-mode prompt optimization |
DSBROWSER_USE_THINKING | false | agent step-by-step thinking (slower, sometimes better) |
DSBROWSER_USE_JUDGE | false | extra judge pass to validate completion (slower, costs more) |
DSBROWSER_ENABLE_PLANNING | false | planner sub-agent for long tasks (slower) |
DSBROWSER_DISABLE_THINKING | |
Evaluation
The offline suite serves a local deterministic website. Three cases are answerable from the DOM; three are answerable only by looking, which is what makes the suite able to tell the vision modes apart — with --vision off those three are expected to fail.
| Case | Capability | Needs vision | Success signal |
|---|
search_and_extract | type, click, extract | no | exact name, price, and path |
navigate_and_submit_form | navigate, fill, select, check, submit | no | deterministic confirmation code |
negative_search | handle empty state | no | zero results reported |
canvas_code | read text drawn on a <canvas> | yes | the voucher code PLUM-4482 |
image_invoice | read text baked into an image | yes | invoice number and total |
chart_reading | interpret a chart rendered as an image | yes | tallest month and its printed value |
# Offline contract tests — no API key, no browser
pytest -q
# Live eval — uses DeepSeek tokens
dsbrowser-eval # current DSBROWSER_VISION_MODE
dsbrowser-eval --vision all # sweep off/auto/on over identical tasks
dsbrowser-eval --vision-only # just the three see-the-page cases
dsbrowser-eval --vision on --no-headless # same suite, headed browser
dsbrowser-eval --repeat 5 --json # five attempts per case
dsbrowser-eval --case canvas_code
python -m dsbrowser.evals
Each run writes raw outputs, errors, timings, per-case token usage and cost, and a per-mode summary to eval-results/run-<timestamp>.json. The summary line reports pass rate split into DOM cases and vision cases, median latency, screenshots sent, and total cost, so the modes can be compared on identical work. Run the suite at least five times before comparing configurations. The local fixture measures agent reliability without public-site drift — keep public-site tasks (like a Hacker News one) as separate realism tests, since network state and page changes make them unsuitable as a regression gate.
Notes
- browser-use disables vision for DeepSeek and this tool undoes it.
agent/service.py still contains if 'deepseek' in self.llm.model.lower(): use_vision = False, written before a DeepSeek vision model existed. DeepSeekBrowserAgent restores the setting on the constructed Agent; without that repair the agent silently never takes a screenshot, whatever you configure.
- Images are legal in
user messages only. DeepSeek answers 400 "Image in system message is unsupported" otherwise, and a single such message kills a whole run. Every request is sanitized before it goes out: images found in system or assistant messages are replaced with a text placeholder.
- Vision needs the vision model. Sending an image to
deepseek-v4-flash returns 400 "This model does not support image", so the configuration is rejected up front with exit code 2 rather than failing mid-run.
- Structured output via
response_format: json_object. DeepSeek's function-calling backend unwraps nested single-key objects, so browser-use's AgentOutput action union ({action_name: params}) came back as bare param dicts that failed pydantic validation. Routing the schema through JSON-object mode (with the schema embedded in the system prompt) makes DeepSeek return the exact expected shape — verified to keep working when the message also carries a screenshot.
- Usage accounting is ours. Upstream
ChatDeepSeek returns usage=None on every path, so this project's DeepSeekBrowserLLM reads DeepSeek's prompt_cache_hit_tokens / prompt_cache_miss_tokens itself and aggregates per run. That is what makes cost_usd and the eval's cost column possible.
- Browser reuse with
keep_alive=True. Agent.run() kills the browser session after every task unless the profile opts out; without this, a shared browser dies after the first task ("CDP client not initialized").
deepseek-v4-flash reasons by default, so DSBROWSER_DISABLE_THINKING=true sends thinking: {"type": "disabled"} on every request via an httpx event hook — measured at ~10x fewer tokens on a trivial call (98 → 10).- browser-use ≥0.13 no longer uses Playwright — it drives a system Chrome/Chromium over CDP. Make sure Chrome is installed.
- Chromium is launched with
--password-store=basic and --use-mock-keychain by default, so headless runs never read or prompt against the (often stale) macOS keychain. Override with if you need real password storage in the profile.
