dsh-vision-fallback
English | 中文
Silent vision enhancement for DeepSeek Harness (dsh): keep your real text-only main model (e.g. deepseek-v4-flash), and let chat images "just work" — every image you drop, paste, or reference in the chat box is automatically sent to a fixed vision model, converted into a factual text observation, and handed to your main model as hidden context. The UI keeps showing your original image; no model groups, no model switching, no extra tools.
Compatible with the current DSH Store window: 0.1.2-alpha.3, 0.1.2-alpha.4, and 0.1.2-alpha.5. Durable MCP / ACP image attachments and nested images forwarded by PTC Mode use the same vision bridge. When enabled, the plugin intentionally keeps advertising image input; otherwise the chat composer rejects the image before it can reach the bridge.
Why
- DeepSeek V4 Flash / Pro and other strong coding models are text-only: dropping an image into the chat box fails with "model does not support image input".
- Existing "vision tool" plugins require saving images as files and invoking a
see_image(path) tool — clunky, and the main model still can't see chat attachments.
- This plugin bridges the gap at the request layer, so chat-box images work exactly like you expect, regardless of which main model you pick in the model picker.
How it works
You drop/paste an image ──► chat attachment (kept visible in UI)
│
▼
agent/pre-step ──► image + current question + recent context
│ │
│ ▼
│ fixed vision model (OpenAI-compatible /chat/completions)
│ │ factual text observation
│ ▼
└──► model-only surface replacement ──► main model (text only)
- The plugin overrides the pre-send capability check, so a text-only model can receive image-bearing messages.
agent/pre-step detects images in the incoming turn, and sends the image, the latest user question, and recent conversation context to the configured vision model.- Your original image stays in the UI as a normal chat attachment.
- A model-only surface replacement swaps the image for the vision observation before the request reaches the main model.
- Switching the main model (DeepSeek, Kimi, MiniMax, ...) never changes the fixed vision model.
MCP / ACP / PTC images
DSH 0.1.0-rc.7 persists images produced by MCP, ACP, and PTC as durable attachments, then exposes them as core image content blocks. This plugin recursively handles images in ordinary messages, tool-result, and nested tool-result content while preserving the original text/image order:
- MCP / ACP durable attachments are read by
attachmentId, not temporary file paths;
- nested PTC subtool images are converted into vision observations;
- multiple images are sent to the vision model in their original order;
- only the model-visible surface is replaced, so the original UI image remains intact.
The image-capability override is an intentional entry-point compatibility layer. It does not claim that the main model has native visual reasoning; it lets the image reach DSH so the bridge can convert it into text context.
DSH compatibility evidence
Verified on 2026-09-03 with Node.js 24.16.0 and a separate disposable DSH_HOME for each release:
| DSH release | Local-path install | --dump-config | Authenticated cold start | Uninstall |
|---|
0.1.2-alpha.3 | passed | passed | HTTP 200 | passed |
0.1.2-alpha.4 | passed | passed | HTTP 200 | passed |
0.1.2-alpha.5 | passed | passed | HTTP 200 | passed |
The Profile operations use the official CLI with plugin --profile web add -w <local-path> and remove -w dsh-vision-fallback. The runtime fix does not mutate the deep-frozen llm/stream request introduced by current DSH builds; compaction creates a copied request and performs one guarded nested dispatch.
Install
From npm / local checkout
# npm (if published) or a local checkout directory
dsh plugin --profile web add dsh-vision-fallback
# or: dsh plugin --profile web add /path/to/dsh-vision-fallback
From source
git clone https://github.com/1HelloMan1/dsh-vision-fallback.git
cd dsh-vision-fallback
pnpm install --config.minimumReleaseAge=0 # pre-release peers may need the release-age flag bypassed
pnpm test # node --test test/*.test.mjs
dsh plugin --profile web add "$PWD"
Then verify and restart:
dsh --profile web --dump-config # expect a "# == dsh-vision-fallback" layer
# restart `dsh web` (patch/bundle layers are not hot-reloaded)
The plugin is also compatible with any profile (headless, TUI) — it registers on the host plane.
Configuration
Two ways, both live (no restart needed after saving):
Web settings page
Open Settings → 视觉增强 / Vision Enhancement in the DSH web UI. It exposes only:
| Field | Default | Meaning |
|---|
| Enabled | true | Master switch |
| Vision model | mimo-v2.5 | OpenAI-compatible model |
| Base URL | https://opencode.ai/zen/go/v1 | The plugin appends /chat/completions |
| Credential ref | OPENCODE_GO_API_KEY | Key resolved from the DSH credential store (never written to env files) |
| Max tokens / Timeout / Max bytes | 1536 / 60000 / 15MB | Vision request limits; oversized images fail |
| Recent context | includeRecentContext: true, contextMessages: 6, contextMaxChars: 6000 | How much recent chat to attach for the vision model |
| Prompt | (Chinese detailed-analysis prompt) | Analysis instruction; the user's question is appended automatically |
| Tag result | true | Prepend 【视觉观察：<model>】 to the observation |
settings.yaml
vision-fallback:
enabled: true
model: mimo-v2.5
baseURL: https://opencode.ai/zen/go/v1
apiKeyRef: OPENCODE_GO_API_KEY
maxTokens: 1536
timeoutMs: 60000
maxBytes: 15728640
includeRecentContext: true
contextMessages: 6
contextMaxChars: 6000
prompt: "请分析这张图片..."
tagResult: true
The API key is resolved through the DSH credentials system (~/.dsh/.credentials.yaml), with process.env[apiKeyRef] as a fallback — it is never materialized into shell environment files by the plugin.
Security & privacy
- The config route is loopback-only and same-origin checked; request bodies are size-limited and schema-validated.
- The vision model gets no tools, no system prompt, no execution permission — only the image, the question, and recent text context.
- Observations are delivered as model-only surface replacements; your original image is never altered in the UI.
- Image reads go through the DSH attachment service (sandbox/observation-policy aware); the vision request carries the official
attributionHeaders().
Usage records (usage.jsonl)
With recordUsage enabled, every real vision call (success or failure) appends one JSON line to
<dshHome>/vision-fallback/usage.jsonl (override via usageLogPath in the settings page), consumed by usage-dashboard.
Fields:
| Field | Meaning |
|---|
ts | Call start time (epoch ms) |
durationMs | Response latency of this call (ms) |
kind | Always "vision" |
status | "ok" success / "error" failure |
model | Vision model name |
inputTokens / outputTokens | Input / output tokens |
cacheReadTokens | Cached input tokens served |
error | Error message (failure entries only) |
imageName / mediaType / imageBytes | Image filename / media type / byte size |
imageIndex / imageTotal | This image's position / total images in the request |
Reusing a remembered observation (observations.json) does not append a line — this file counts real external vision calls only.
Observation cache semantics
Observations are keyed by the image's occurrence in a session (session id + message id):
- The same image at the same message position processed again (restart recovery, replay, compaction) → reused, no re-recognition;
- The same image at a new position in the session (new turn, main model asking to "look again") → re-recognized with fresh context;
- Observations are not shared across sessions.
The cache is capped at 256 entries (LRU eviction); failed results are never cached.
About image compression
In the current version, “compression” means reusing an existing vision observation during conversation-context compaction; it does not re-run vision analysis. The plugin does not yet resize, transcode, or quality-compress image files. Images above maxBytes fail explicitly instead of being silently changed. A future real image-compression feature should report the before/after byte counts and media type.
Default vision route
- Model:
mimo-v2.5 · Endpoint: https://opencode.ai/zen/go/v1/chat/completions · Credential: OPENCODE_GO_API_KEY
Any OpenAI-compatible vision endpoint works (Zhipu GLM-4V-Flash, SiliconFlow Qwen-VL, vLLM, Ollama, ...) — just change model, baseURL, and apiKeyRef in the settings page.
Relationship to the OpenCode ecosystem
The OpenCode community opencode-see-image hands a filePath + task question to a fixed vision model and returns text to the main model. DSH additionally performs a pre-send image-capability check, which this plugin also overrides, using DSH's official agent/pre-step and model-only surface replacement to keep the UI silent.
Development
pnpm test # node --test test/*.test.mjs
Structure:
dsh-vision-fallback/
├── package.json # dsh.bundle + dsh.client manifests
├── cordis.patch.yml # inserts the vision-fallback row
├── lib/index.js # host plugin (pre-step bridge, config route, controller)
├── lib/client.js # Web settings page ("视觉增强")
└── test/ # unit tests
License
MIT
