dsh-vision-pro-bridge
Give a text-only DeepSeek-V4-Pro brain real image understanding — without switching models, without Ollama, and without a second API key.
DeepSeek Harness (dsh) plugin. English · 简体中文
The problem
deepseek-v4-pro (and deepseek-v4-flash) are text-only models: their API rejects image bytes. In DeepSeek Harness, pasting a screenshot into a Pro session fails with "does not accept image input". Your only official option is to switch the whole conversation to the weaker, Flash-tier deepseek-v4-flash-vision-exp.
What this plugin does
It registers a twin provider route — deepseek-vision-pro (model deepseek-v4-pro-vision) — that advertises image support. When you attach an image, the plugin first asks deepseek-v4-flash-vision-exp to transcribe it into text (using your existing DEEPSEEK_API_KEY), then hands that text — plus a stable local file path — to deepseek-v4-pro. The DeepSeek wire route never sees an image; V4-Pro keeps its coding brain and still "sees" the picture.
paste screenshot → [image block admitted]
→ transcribe via deepseek-v4-flash-vision-exp (text)
→ text + local path → deepseek-v4-pro answers
Why this one
- Zero third-party dependencies. It reuses only the harness's own
@deepseek-ai/dsh-llm and @deepseek-ai/dsh-llm-deepseek. No openai, no sharp, no schemastery, no Ollama.
- DeepSeek-only, single provider. Images go only to DeepSeek (
deepseek-v4-flash-vision-exp), through the same DEEPSEEK_API_KEY. No second VLM vendor, no local model, no anonymous endpoints.
- Official image pipeline, byte-for-byte. It subclasses the official
DeepSeekAdapter, so image normalization, Files-API upload, and inline-base64 fallback are identical to official behavior — no hand-rolled HTTP.
- Minimal & auditable. One file, ~250 lines. Easy to read, verify, and fork.
- Content-addressed persistence. Images are saved under
~/.dsh/vision-pro-bridge/images/ and the stable local path is passed to the model, so V4-Pro can reference it.
- Pro-focused. One model,
deepseek-v4-pro-vision: keep V4-Pro's coding ability, add vision.
Install
dsh plugin --profile web add dsh-vision-pro-bridge
# restart dsh web
Usage
- In the model picker, choose DeepSeek-V4-Pro (视觉桥).
- Paste or drag a screenshot and ask normally.
Make it the default for new sessions (~/.dsh/settings.yaml):
agent-default-model:
provider: deepseek-vision-pro
model: deepseek-v4-pro-vision
Configuration
Environment variables (all optional):
| Variable | Default | Purpose |
|---|
DEEPSEEK_API_KEY | credentials service | DeepSeek key (same as the official route) |
DEEPSEEK_BASE_URL | https://api.deepseek.com | Endpoint (same as the official route) |
DSH_VISION_PRO_BRIDGE_PROMPT | built-in English transcription prompt | Override the transcription prompt (e.g. Chinese UI analysis) |
Images are cached content-addressed under ~/.dsh/vision-pro-bridge/images/ (re-attaching the same image reuses the transcription); an activity log is written to ~/.dsh/vision-pro-bridge-activity.jsonl.
How it works
apply(ctx) registers a DeepSeekAdapter subclass for the provider route deepseek-vision-pro.listModels / resolveModel / prepareCall all declare inputModalities: ['text', 'image'], so the harness's image gates (paste admission, thumbnails, read_image) open. Overriding prepareCall is required — otherwise the LLM runtime projects images to placeholders before they ever reach the adapter.stream() walks every message (including nested tool-result content), and for each image block reads its bytes, saves them content-addressed, and calls super.stream() against deepseek-v4-flash-vision-exp to produce a text transcription.- The image block is replaced with
[图片（视觉桥分析 by …）]\n<transcription>\n\n本地路径：<path> and the now text-only conversation is streamed to deepseek-v4-pro.
With no image, stream() forwards directly to Pro — zero overhead.
Limitations
- Transcription is lossy: great for layout / OCR / UI review, not for exact pixel coordinates.
- The transcription model is fixed to
deepseek-v4-flash-vision-exp.
- The endpoint is resolved from
DEEPSEEK_BASE_URL (not the settings-page llm-deepseek overrides).
- No fallback chain: if the transcription call fails, the image becomes a placeholder and the turn continues.
Alternatives
For a feature-rich take (fallback VLM chain, local Ollama, settings-page UI, image downscaling, classified errors), see dsh-vision-proxy. This plugin is the minimal, DeepSeek-only, zero-dependency version of the same idea.
License
MIT
