@dsh-extension/dsh-vision-bridge
适用于纯文本 DSH 会话的按需视觉功能：图像会变为标记，并由 vision_describe 工具仅将图像和问题发送到兼容 OpenAI 的视觉模型
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add @dsh-extension/dsh-vision-bridge@1.0.0Vision Bridge 以 @dsh-extension/dsh-vision-bridge 发布，当前版本为 1.0.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
On-demand vision for text-only DeepSeek Harness (DSH) sessions.
A DSH plugin that gives a text-only DeepSeek session on-demand multimodal capability: the session stays on its text model for every turn, and only when the model actually needs to look at pixels — a screenshot, an uploaded image, a diagram, a chart — does it call the vision_describe tool, which sends only the image(s) + a focused question to an OpenAI-compatible vision model.
/v1/chat/completions service (OpenAI, DeepSeek, Gemini proxy, local vLLM/One-API, …).Want to appear here? — sponsor this project with an API donation.
| xiaoyaoapi | 🎉 Thanks to xiaoyaoapi for donating their API to this project! xiaoyaoapi is an OpenAI-compatible AI API aggregation gateway for developers, built on New API with a unified admin dashboard. It offers unified key management, transparent usage tracking, and multi-channel access to mainstream large models under a single endpoint — letting developers integrate leading LLM services at lower cost and with greater convenience, ready to use as the vision endpoint of this plugin. |
User / tool produces an image ──► image stays in the session and UI
│
▼ (model input layer)
image is rewritten to a text marker
(marker carries the attachment id and hints
the model to call vision_describe)
│
▼
text model calls vision_describe(attachmentIds / paths, question)
│
▼
vision model (receives only image + question) → text answer
→ returned as a normal tool result
agent/pre-step hook records every image attachment that appears in the session (user uploads and tool-produced screenshots, including ones nested inside tool-result), building an attachment index that vision_describe uses to resolve bytes by id.session.deriveMessages() is wrapped so that no text-model request ever contains image blocks (the native DeepSeek adapter rejects them); images are replaced by text markers. The session event log and UI keep showing the original images.llm-pi-ai builds the OpenAI multimodal request; the plugin only maintains a single vision-bridge provider route and does not re-implement a protocol adapter.DSH Web runs an image-capability check before a message enters the agent, based on the current DeepSeek model. This plugin keeps an admission bypass so images can enter the session first; the marker rewrite then guarantees no text-model request carries image blocks. When the plugin is disabled or uninstalled, the native admission check is restored.
Check GET /_dsh/vision-bridge/settings for the live value.admissionBypass and dependency-service status.
Install from the npm registry (not a local checkout) — one command:
# if you already have the `dsh` CLI on PATH:
dsh plugin --profile web add @dsh-extension/dsh-vision-bridge
# or, if you have been using npx all along:
npx @deepseek-ai/dsh@0.1.0-rc.6 plugin --profile web add @dsh-extension/dsh-vision-bridge
The
--profileflag targets the profile you boot (webis the browser UI profile). Omit it or adapt it if your profile has a different name.After a new client bundle is added, restart
dsh webonce so the UI picks it up.
Configure it in Settings → Vision Bridge (DSH Web), or edit ~/.dsh/vision-bridge.json:
{
"enabled": true,
"baseUrl": "https://api.openai.com/v1",
"apiKey": "sk-xxxx",
"apiKeyEnv": "",
"model": "gpt-5.6-terra"
}
baseUrl accepts an API root, a .../v1 base, or a full .../chat/completions URL (the plugin normalizes it).apiKey and apiKeyEnv are mutually exclusive. A directly entered key is synced to the DSH credential store and referenced as DSH_VISION_BRIDGE_API_KEY.vision-bridge route inside DSH's llm-pi-ai.providers and never touches other providers.enabled: false disables the whole chain: no tool registration, no image rewriting, no admission bypass (native behavior restored).Precedence (highest wins): Settings page (with schema defaults) → environment variables → config file.
Environment overrides: DSH_VISION_BRIDGE_BASE_URL, DSH_VISION_BRIDGE_API_KEY, DSH_VISION_BRIDGE_API_KEY_ENV, DSH_VISION_BRIDGE_MODEL, DSH_VISION_BRIDGE_ENABLED.
vision_describe toolattachmentIds: image attachment ids from the current conversation (shaped like sha256:...), one or several;paths: absolute local image file paths (png/jpeg/webp/gif) — use either or both, 1–4 images in total;question: required — a focused, specific question about the image(s).vision-bridge route → returns the text answer as a tool result.paths go through DSH's sandbox-aware file service.npm test
The suite covers: image-marker rewriting (including nested tool-result), both id- and path-based resolution, event-log attachment indexing, full-chain shutdown when disabled, and text-only sessions staying untouched.
From a local checkout:
dsh plugin inject /path/to/dsh-vision-bridge
deepseek · deepseek-harness · dsh · plugin · vision · multimodal · vision-language-model · VLM · image understanding · screenshot · OCR · image analysis · OpenAI-compatible · text-only-llm · on-demand vision · LLM agent