dsh-tool-accurate-vision
面向模型的 accurate_vision 工具：通过视觉模型进行精确的图像空间推理
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add github:imkingjh999/dsh-tool-accurate-vision#d6b117506175c210e1774a1a1cb94283e6be3b02Tool Accurate Vision 以 dsh-tool-accurate-vision 发布，当前版本为 0.1.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
Model-facing accurate_vision tool for
DeepSeek Harness:
precise spatial reasoning over an image file via an OpenAI-compatible vision model.
Ported from pi-accurate-vision.
A vision model reads the image and returns a structured note plus
bounding-box primitives normalised to 0–1000; this tool formats them as a
<vision-context> block the next model turn reads — giving a text-only agent
exact object positions, layout, and OCR without losing spatial fidelity.
English | 中文
dsh plugin --profile web add dsh-tool-accurate-vision
Or from source:
dsh plugin --profile web add github:your-username/dsh-tool-accurate-vision
Set the vision API key (separate from DEEPSEEK_API_KEY):
export VISION_API_KEY=sk-...
image file ──► base64 data URL ──► vision chat/completions ──► JSON note + primitives
│
<vision-context> XML ──► next model turn
The pure vision core (src/bridge.ts) is provider-agnostic:
any OpenAI-compatible multimodal chat/completions endpoint works.
The Cordis host (src/index.ts) owns config, credential
resolution, and the registered tool.
Every call also writes a self-contained SVG — the original image with every
bounding box and label drawn on it — returned as the annotatedImage path,
so the boxes can be eyeballed instead of trusted blind
(set annotate: false to skip it).
Ask an image question with a checkable answer — in this hand-drawn physicists network, which node sits physically closest to 居里夫人 (Marie Curie), ignoring the connecting lines? — and the gap between plain vision and this tool becomes measurable. The test image is the aged network diagram below:
The test image: a hand-drawn physicists network
Asking a multimodal model directly yields a visual impression, not a measurement: "郎之万, at the lower left, looks closest" — nothing to verify, and as it turns out, wrong.
Vision text without structured primitives can be worse than no numbers at all: the model invents plausible-looking coordinates in prose, then contradicts itself — a claimed ~15-unit gap while its own two boxes imply 59 — and returns the same wrong answer.
With this tool's normalised primitives, every node carries a checkable 0–1000 bounding box, so the agent computes real edge-to-edge distances in code: 皮卡尔德 25.96 vs 郎之万 58.00. The correct answer — 皮卡尔德 (Piccard) — arrives with the numbers that prove it.
That is the core advantage: bounding-box primitives turn visual impressions
into geometry. Positions, distances, and layout become facts a text-only agent
can compute and verify, not guesses it has to trust. For distance questions the
canonical edge-to-edge computation pairs the facing edges per axis
(dx = max(a.x1 - b.x2, b.x1 - a.x2, 0), same for y, then hypot); the
tested helper bboxEdgeDistance(a, b) ships with this package so downstream
agents never pair the wrong edges.
Override in your profile's cordis.patch.yml:
- id: tool-accurate-vision
config:
model: gpt-4o # any OpenAI-compatible multimodal model
baseURL: https://api.openai.com/v1
apiKeyEnv: VISION_API_KEY # credential reference
primitives: true # request bounding-box primitives
annotate: true # also write an SVG with boxes drawn on the image
maxTokens: 8192
timeoutSecs: 120
temperature: 0
disableThinking: true # skip the reasoning phase (MiniMax): faster & steadier
Faithful port of pi-accurate-vision (which itself extracted DeepSeek-TUI's
crates/tui/src/vision/bridge.rs). The parsing, prompt, and formatting logic
is preserved verbatim; only the host integration targets the Cordis ctx.tools
registry with schemastery config and the credentials seam.
MIT