@try-works/dsh-vision-worker
A DeepSeek Harness plugin that gives text-only models real vision.
When a caller cannot see images, dsh-vision-worker routes the image to
Cloudflare Workers AI @cf/moonshotai/kimi-k2.6 (vision-capable) and returns a
deterministic, versioned envelope — so the text-only caller can act on
structured facts instead of guessing from prose.
The problem
- Text-only models reject image content, so they cannot read a website
screenshot, a photo, a chart, or a scanned table.
- Handing them raw vision prose forces re-parsing and loses structure.
The fix
One call returns a righthand.vision.v1 envelope:
{
"schema": "righthand.vision.v1",
"model": "@cf/moonshotai/kimi-k2.6",
"status": "ok",
"summary": "...",
"answer": "...",
"content": {
"text": [{"value": "...", "confidence": 0.97, "bbox": [x,y,w,h]}],
"measurements": [{"name": "...", "value": 12, "unit": "%"}],
"tables": [{"name": "...", "columns": [...], "rows": [[...]]}],
"code": [{"language": "...", "text": "..."}],
"entities": [{"type": "...", "value": "..."}]
},
"actions": ["verify the Q3 total against the source table"],
"confidence": 0.9,
"warnings": [],
"images": []
}
Install
dsh plugin --profile <profile> add @try-works/dsh-vision-worker
Or apply the overlay directly:
dsh --profile <profile> --patch ./cordis.patch.yml
Configure
Set one of these transports (the plugin tries workerUrl first, then
accountId + apiTokenRef):
# cordis.patch.yml
- insert:
- id: dsh-vision-worker
name: '@try-works/dsh-vision-worker'
config:
# Deployed worker (see cloud/):
workerUrl: 'https://vision-worker.<you>.workers.dev'
# …or Workers AI REST directly:
# accountId: '<your Cloudflare account id>'
# apiTokenRef: 'CLOUDFLARE_API_TOKEN'
# model: '@cf/moonshotai/kimi-k2.6'
# locale: 'en'
Store the token with the harness credential provider (never inline):
rh_credential_set ref=CLOUDFLARE_API_TOKEN value=<token>
Tools
| Tool | Purpose |
|---|
vw_analyze | Analyze image(s) with a prompt → envelope + callerText |
vw_ask | Follow-up question about the same image(s) |
vw_status | Report active transport/model without exposing the token |
vw_locale | Get/set UI language (en or zh-CN) |
Tool names are stable identifiers; descriptions and rendered status text
localize live via the locale setting.
Deploy the worker
cd cloud
npx wrangler deploy
cloud/index.js is a zero-build Worker exposing POST /analyze (plus
GET /health). It takes { prompt, images, system? } and returns the same
righthand.vision.v1 envelope.
Test
pnpm install
pnpm test # 20 tests: core, schema, transport, tools, fixtures
pnpm typecheck
node --experimental-strip-types test.ts # inside cloud/ — worker handler test
Routing architecture
The observable Workers AI routing protocol (name→effective-model
aliasing, task-family dispatch, result-envelope shapes, gating) is
documented in docs/workers-ai-routing-protocol.md,
derived from live probes of every cataloged model.
Layout
src/core/vision.ts — capability routing, multimodal input, normalization,
multi-step (transport injected; zero deps).src/core/schema.ts — the envelope format, prompt template, tolerant parser,
and callerText renderer.src/transport.ts — Workers AI REST client + deployed-worker client.src/vision-tools.ts — the DSH-native tool surface (ctx.tools/
ctx.settings/ctx.credentials).src/i18n.ts — EN + zh-CN string tables.src/fixture.ts — deterministic PNG fixtures (real bytes, no deps).cloud/ — deployable Worker.
