dsh-llm-vision

Llm Vision

面向模型的 describe_image + extract_text 工具，适用于 DeepSeek Harness Web GUI：通过兼容 OpenAI 的视觉端点，为仅支持文本的模型提供可靠的图像理解和 OCR，并支持关键检查提示、自动预处理、重试以及持久化答案缓存。

目标 Profile

插件会安装到这里；不确定时保持 web。

npx -y @deepseek-ai/dsh plugin --profile web add github:1710782766/dsh-llm-vision#a4a5272448c1b6df7d90b7b7afc6170e26c2ef43

兼容性与来源证明

Llm Vision 以 dsh-llm-vision 发布，当前版本为 0.3.2。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。

DSH 兼容范围: *
运行环境: web
发布来源: github
Registry 更新时间: 2026/8/31

版本

0.3.2stable

2026/8/31

0.3.0stable

2026/8/24

README

dsh-llm-vision

English | 中文

Give your DeepSeek Harness a pair of eyes — reliable image understanding and OCR for text-only models, configured entirely in the GUI.

Paste an image and the model describes or reads it; big screenshots are auto-compressed, transient failures retry, identical images hit a persistent cache. Built-in free presets (Zhipu / Gemini / DashScope) get you running without touching a config file.

Quick start

dsh plugin --profile web add dsh-llm-vision@0.3.2

Install with the command above (or see Install).
Restart the GUI once — plugins load at boot, so the card is not visible until then. Configuration changes after install never need a restart.
Open Settings → Plugins → llm-vision and pick a Provider preset: zhipu, gemini, or dashscope fill the endpoint fields for you — free routes, no payment details (see the free presets table).
Paste your API key into the card's API key field and Save — it is stored in your owner-only settings document and never shown again.
Use it — paste / drag / drop an image into the composer and send; the model now sees it. Or call the llm_vision_check tool for a full pipeline diagnosis.

Why

Text-only models (DeepSeek V4, GLM text series, …) cannot see images. This plugin registers model-facing tools backed by any OpenAI-compatible vision endpoint:

Tool	Purpose
`describe_image`	Image understanding with two perspectives: normal (natural description) and critical (objective inspection that actively reports text misalignment, overlap, occlusion, wrapping anomalies, missing elements, and separates fact from guess). The critical lens is the antidote to vision models rationalizing rendering bugs — use it for page/UI problem reports and screenshot-vs-design comparisons. Accepts a single `image` or a batch of up to 8 `images` read together in one call.
`extract_text`	OCR & document parsing through a dedicated OCR model — ID cards, invoices, receipts; structured output (JSON/CSV) on request; verbatim extraction that never guesses missing text.
`llm_vision_check`	Diagnostics: verifies the configuration, that an API key resolves, and that the endpoint answers an authenticated probe — optionally with a real end-to-end vision call (`testCall`). The key itself never appears in the report.

Plus the DSH-native experience:

Paste / drag / drop images into the composer and send — the browser half rewrites the image-bearing send into attach references the text model can resolve, and upgrades the references into inline thumbnails in the transcript.
Live settings card (Settings → Plugins → llm-vision): endpoint, models, prompts, bounds, retries, preprocessing, and cache — saves apply to the very next call.
Three input kinds per call: local absolute path, http(s) URL (redirects refused), or attachment reference.
The image never enters the session log — only the returned text crosses into the conversation.

Install

dsh plugin --profile web add dsh-llm-vision@0.3.2

Then restart the GUI once — plugins load at boot, so the plugin and its settings card become visible only after the restart (configuration changes after that never need one).

The version is pinned on purpose: pnpm 11 holds back packages published in the last 24 hours, so a bare add dsh-llm-vision (latest) would silently install the previous release on launch day. This line is bumped with every release. --profile web is the GUI profile of this deployment — use your own profile name if it differs.

Requires dsh ≥ 0.1.2-alpha.1 — the settings-card host API and the browser-half store moved in that release; older harness builds cannot serve the card.

From a source checkout the same command accepts a tarball or local path (pnpm pack names the tarball after the current version — use that name):

pnpm install && pnpm build && pnpm pack   # → dsh-llm-vision-<version>.tgz
dsh plugin --profile web add ./dsh-llm-vision-<version>.tgz
# or: dsh plugin --profile web add /path/to/dsh-llm-vision   (build first — lib/ is gitignored)

The tarball ships prebuilt lib/ (both the node half and lib/client.js), so no build step runs on the installing machine.

Configure

Everything is configured in the GUI — the Settings → Plugins → llm-vision card. No patch file, no environment exports required:

Open Settings → Plugins and find the llm-vision card.
Pick a Provider preset for a zero-config route, or set baseURL / model / ocrModel yourself (custom).
Paste the API key into the card's API key field — the simple path: it is stored in the harness's owner-only settings document (~/.dsh/settings.yaml, 0600) and never shown again. Advanced: leave the field empty and let apiKeyEnv resolve through the credential seam instead (presets prefill e.g. DASHSCOPE_API_KEY; the default is VISION_API_KEY) — for users who prefer environment variables.
Save — the change reaches the very next tool call, no restart.

Before configuring, the first call fails with a clear hint (llm-vision: baseURL must be an absolute http(s) URL) — that is the expected unconfigured state, not a broken install.

The values live in the harness settings document (~/.dsh/settings.yaml, 0600, shared across profiles) and are written by the GUI. A profile patch layer may still provide deployment defaults for the card (shown as "Inherit"), but the card's saved values always win — the GUI is the only configuration surface a user needs. A deployment without a settings provider falls back to the built-in defaults.

Free presets (zero-cost routes)

The Provider preset selector fills baseURL / model / ocrModel / apiKeyEnv for you — free routes, no payment details. Free policies change, so re-check the provider docs if a call stops working:

Preset	Endpoint	Getting a free key
`zhipu`	Zhipu BigModel — permanently free GLM-4V-Flash; the best default in mainland China	open.bigmodel.cn — register, create an API key; free tier, no card
`gemini`	Google Gemini — free key from Google AI Studio (aistudio.google.com)	AI Studio → "Get API key", no card; not reachable from mainland China without a proxy
`dashscope`	Alibaba DashScope (the default models) with free quota	Alibaba Cloud Bailian console (bailian.console.aliyun.com) — free quota; reachable from mainland China

Picking a preset prefills the endpoint fields (still editable before saving); explicit field values always win at call time. The free presets reuse the vision model for OCR (extract_text drives it with the OCR prompt) — free tiers are rate-limited, so they suit interactive use better than batch runs.

Key	Default	Meaning
`provider`	`custom`	Endpoint preset: `custom` (all fields explicit), `dashscope`, `zhipu` (free GLM-4V-Flash), or `gemini` (free key). Explicit fields win.
`baseURL`	— (required for `custom`)	OpenAI-compatible root URL; `/chat/completions` or `/responses` appended per `apiStyle`.
`model`	preset, else `qwen3-vl-plus`	Vision model for `describe_image`; optional thinking suffix `:off/:low/:medium/:high`.
`ocrModel`	preset, else `qwen3.5-ocr`	OCR model for `extract_text`; same suffix support.
`apiKey`	—	Inline key, stored in the settings document (secret: never shown by the GUI).
`apiKeyEnv`	`VISION_API_KEY`	Credential-reference (env var name) resolved through the credential seam; empty disables.
`criticalPrompt`	built-in	`describe_image` critical-perspective prompt when the model passes none.
`normalPrompt`	built-in	`describe_image` normal-perspective prompt when the model passes none.
`ocrPrompt`	built-in	`extract_text` prompt when the model passes none.
`apiStyle`	`chat-completions`	`chat-completions` or `responses`.
`maxBytes`	`10485760`	Image byte bound (local files and downloads). Hi-res PNG wallpapers (10–30 MB) exceed the default; raise it — preprocessing compresses after loading.
`maxOutputTokens`	`1024`	Output-token cap sent to the endpoint.
`timeoutMs`	`60000`	Per-attempt timeout.

Reliability engineering

Auto-preprocessing — images over 1568px are scaled, oversize files re-encoded (JPEG q85, transparent formats kept as PNG) via the macOS built-in sips; every failure silently falls back to the original image. HEIC/HEIF inputs are always re-encoded to JPEG (endpoints support HEIC unevenly), failing loudly only when sips is absent. Fixes the classic "big screenshot times out" failure.
Retries — transient errors retry up to maxRetries with exponential backoff (≤ 4s) under a shrinking per-attempt budget (total ≤ 2× timeout). Exhausted retries append （已重试 N 次）. Caller cancellation aborts immediately without retry.
Persistent cache — identical image + model + prompt + preprocessing settings hit a content-addressed cache (SHA-256 over the image bytes) at ~/.cache/dsh-llm-vision/; only the text answer is stored, never image bytes; TTL 30 days, 500 entries, atomic writes, 0600/0700 permissions. Note: OCR results of sensitive documents are stored in plain text there — set cacheEnabled to false when that matters.

Security model

The vision request and any image download refuse HTTP redirects (redirect: 'error') — bearer credentials and image bytes never leave the configured endpoint.
Request bodies carry the base64 image but never the key; parsed credentials are never logged.
Only http(s) URLs and local paths are accepted; all other schemes are rejected.
Attach uploads are validated (strict base64, magic bytes, byte bound) before the attachment store persists them; only the reference JSON (text) enters the session.
Response bodies are capped (maxOutputTokens × 8 + 64 KiB) before parsing; error excerpts are bounded to 200 chars.
Calling the tools sends the image bytes to the configured endpoint — only hand the model images you are comfortable leaving your machine.

Testing status

A fully offline test suite (vitest, mock HTTP server, tmp-dir cache), a strict typecheck, and CI on every push. Verified end-to-end in the real DSH web GUI against a live OpenAI-compatible vision endpoint: describe_image reads a real image (DashScope qwen3-vl-plus), extract_text OCR returns real transcription (qwen3.5-ocr), the attach upload/readback routes work through the live web server, and the settings card renders and saves in the plugin-configuration page — see Configure.

Known limitations

Attachment/upload channel: PNG / JPEG / GIF / WebP only (the official attachment store's type set). HEIC/HEIF images are read directly by the tools from local paths and URLs — preprocessing re-encodes them to JPEG on macOS — but pasting a HEIC file into the GUI is rejected with a hint; convert it or pass the path instead. On Windows/Linux (no sips) a HEIC/HEIF read fails with a clear message.
Preprocessing relies on macOS sips (zero dependencies); on Windows/Linux the plugin degrades silently and sends the original bytes — never an error, but oversized images are then likelier to time out. The bound gates loading (maxBytes), so a too-small bound is a clean rejection, never a crash.

Development

pnpm typecheck   # tsc -b + vitest program
pnpm test        # vitest run (fully offline)
pnpm build       # tsc -b && tsdown → lib/ + lib/client.js
pnpm watch       # tsdown --watch

License & attribution

Apache-2.0. Built on: deepseek-harness packages/vision/tool-describe-image (whitelonng/dsh-plugin-describe-image, MIT), the dsh-web-ui plugin family (Apache-2.0), and the llm_vision design (MIT). See NOTICE and AGENTS.md.

README

dsh-llm-vision

English | 中文

Give your DeepSeek Harness a pair of eyes — reliable image understanding and OCR for text-only models, configured entirely in the GUI.

Quick start

dsh plugin --profile web add dsh-llm-vision@0.3.2

Install with the command above (or see Install).
Restart the GUI once — plugins load at boot, so the card is not visible until then. Configuration changes after install never need a restart.
Open Settings → Plugins → llm-vision and pick a Provider preset: zhipu, gemini, or dashscope fill the endpoint fields for you — free routes, no payment details (see the free presets table).
Paste your API key into the card's API key field and Save — it is stored in your owner-only settings document and never shown again.
Use it — paste / drag / drop an image into the composer and send; the model now sees it. Or call the llm_vision_check tool for a full pipeline diagnosis.

Why

Text-only models (DeepSeek V4, GLM text series, …) cannot see images. This plugin registers model-facing tools backed by any OpenAI-compatible vision endpoint:

Tool	Purpose
`describe_image`	Image understanding with two perspectives: normal (natural description) and critical (objective inspection that actively reports text misalignment, overlap, occlusion, wrapping anomalies, missing elements, and separates fact from guess). The critical lens is the antidote to vision models rationalizing rendering bugs — use it for page/UI problem reports and screenshot-vs-design comparisons. Accepts a single `image` or a batch of up to 8 `images` read together in one call.
`extract_text`	OCR & document parsing through a dedicated OCR model — ID cards, invoices, receipts; structured output (JSON/CSV) on request; verbatim extraction that never guesses missing text.
`llm_vision_check`	Diagnostics: verifies the configuration, that an API key resolves, and that the endpoint answers an authenticated probe — optionally with a real end-to-end vision call (`testCall`). The key itself never appears in the report.

Plus the DSH-native experience:

Paste / drag / drop images into the composer and send — the browser half rewrites the image-bearing send into attach references the text model can resolve, and upgrades the references into inline thumbnails in the transcript.
Live settings card (Settings → Plugins → llm-vision): endpoint, models, prompts, bounds, retries, preprocessing, and cache — saves apply to the very next call.
Three input kinds per call: local absolute path, http(s) URL (redirects refused), or attachment reference.
The image never enters the session log — only the returned text crosses into the conversation.

Install

dsh plugin --profile web add dsh-llm-vision@0.3.2

Then restart the GUI once — plugins load at boot, so the plugin and its settings card become visible only after the restart (configuration changes after that never need one).

Requires dsh ≥ 0.1.2-alpha.1 — the settings-card host API and the browser-half store moved in that release; older harness builds cannot serve the card.

From a source checkout the same command accepts a tarball or local path (pnpm pack names the tarball after the current version — use that name):

pnpm install && pnpm build && pnpm pack   # → dsh-llm-vision-<version>.tgz
dsh plugin --profile web add ./dsh-llm-vision-<version>.tgz
# or: dsh plugin --profile web add /path/to/dsh-llm-vision   (build first — lib/ is gitignored)

The tarball ships prebuilt lib/ (both the node half and lib/client.js), so no build step runs on the installing machine.

Configure

Everything is configured in the GUI — the Settings → Plugins → llm-vision card. No patch file, no environment exports required:

Open Settings → Plugins and find the llm-vision card.
Pick a Provider preset for a zero-config route, or set baseURL / model / ocrModel yourself (custom).
Paste the API key into the card's API key field — the simple path: it is stored in the harness's owner-only settings document (~/.dsh/settings.yaml, 0600) and never shown again. Advanced: leave the field empty and let apiKeyEnv resolve through the credential seam instead (presets prefill e.g. DASHSCOPE_API_KEY; the default is VISION_API_KEY) — for users who prefer environment variables.
Save — the change reaches the very next tool call, no restart.

Before configuring, the first call fails with a clear hint (llm-vision: baseURL must be an absolute http(s) URL) — that is the expected unconfigured state, not a broken install.

Free presets (zero-cost routes)

Preset	Endpoint	Getting a free key
`zhipu`	Zhipu BigModel — permanently free GLM-4V-Flash; the best default in mainland China	open.bigmodel.cn — register, create an API key; free tier, no card
`gemini`	Google Gemini — free key from Google AI Studio (aistudio.google.com)	AI Studio → "Get API key", no card; not reachable from mainland China without a proxy
`dashscope`	Alibaba DashScope (the default models) with free quota	Alibaba Cloud Bailian console (bailian.console.aliyun.com) — free quota; reachable from mainland China

Key	Default	Meaning
`provider`	`custom`	Endpoint preset: `custom` (all fields explicit), `dashscope`, `zhipu` (free GLM-4V-Flash), or `gemini` (free key). Explicit fields win.
`baseURL`	— (required for `custom`)	OpenAI-compatible root URL; `/chat/completions` or `/responses` appended per `apiStyle`.
`model`	preset, else `qwen3-vl-plus`	Vision model for `describe_image`; optional thinking suffix `:off/:low/:medium/:high`.
`ocrModel`	preset, else `qwen3.5-ocr`	OCR model for `extract_text`; same suffix support.
`apiKey`	—	Inline key, stored in the settings document (secret: never shown by the GUI).
`apiKeyEnv`	`VISION_API_KEY`	Credential-reference (env var name) resolved through the credential seam; empty disables.
`criticalPrompt`	built-in	`describe_image` critical-perspective prompt when the model passes none.
`normalPrompt`	built-in	`describe_image` normal-perspective prompt when the model passes none.
`ocrPrompt`	built-in	`extract_text` prompt when the model passes none.
`apiStyle`	`chat-completions`	`chat-completions` or `responses`.
`maxBytes`	`10485760`	Image byte bound (local files and downloads). Hi-res PNG wallpapers (10–30 MB) exceed the default; raise it — preprocessing compresses after loading.
`maxOutputTokens`	`1024`	Output-token cap sent to the endpoint.
`timeoutMs`	`60000`	Per-attempt timeout.

Reliability engineering

Auto-preprocessing — images over 1568px are scaled, oversize files re-encoded (JPEG q85, transparent formats kept as PNG) via the macOS built-in sips; every failure silently falls back to the original image. HEIC/HEIF inputs are always re-encoded to JPEG (endpoints support HEIC unevenly), failing loudly only when sips is absent. Fixes the classic "big screenshot times out" failure.
Retries — transient errors retry up to maxRetries with exponential backoff (≤ 4s) under a shrinking per-attempt budget (total ≤ 2× timeout). Exhausted retries append （已重试 N 次）. Caller cancellation aborts immediately without retry.
Persistent cache — identical image + model + prompt + preprocessing settings hit a content-addressed cache (SHA-256 over the image bytes) at ~/.cache/dsh-llm-vision/; only the text answer is stored, never image bytes; TTL 30 days, 500 entries, atomic writes, 0600/0700 permissions. Note: OCR results of sensitive documents are stored in plain text there — set cacheEnabled to false when that matters.

Security model

The vision request and any image download refuse HTTP redirects (redirect: 'error') — bearer credentials and image bytes never leave the configured endpoint.
Request bodies carry the base64 image but never the key; parsed credentials are never logged.
Only http(s) URLs and local paths are accepted; all other schemes are rejected.
Attach uploads are validated (strict base64, magic bytes, byte bound) before the attachment store persists them; only the reference JSON (text) enters the session.
Response bodies are capped (maxOutputTokens × 8 + 64 KiB) before parsing; error excerpts are bounded to 200 chars.
Calling the tools sends the image bytes to the configured endpoint — only hand the model images you are comfortable leaving your machine.

Testing status

Known limitations

Attachment/upload channel: PNG / JPEG / GIF / WebP only (the official attachment store's type set). HEIC/HEIF images are read directly by the tools from local paths and URLs — preprocessing re-encodes them to JPEG on macOS — but pasting a HEIC file into the GUI is rejected with a hint; convert it or pass the path instead. On Windows/Linux (no sips) a HEIC/HEIF read fails with a clear message.
Preprocessing relies on macOS sips (zero dependencies); on Windows/Linux the plugin degrades silently and sends the original bytes — never an error, but oversized images are then likelier to time out. The bound gates loading (maxBytes), so a too-small bound is a clean rejection, never a crash.

Development

pnpm typecheck   # tsc -b + vitest program
pnpm test        # vitest run (fully offline)
pnpm build       # tsc -b && tsdown → lib/ + lib/client.js
pnpm watch       # tsdown --watch

Llm Vision

兼容性与来源证明

版本

相关插件

Llm Vision

README

dsh-llm-vision

Quick start

Why

Install

Configure

Free presets (zero-cost routes)

Reliability engineering

Security model

Testing status

Known limitations

Development

License & attribution

兼容性与来源证明

版本

相关插件

相关插件

相关插件

README

dsh-llm-vision

Quick start

Why

Install

Configure

Free presets (zero-cost routes)

Reliability engineering

Security model

Testing status

Known limitations

Development

License & attribution