KoboldCpp for DeepSeek Harness
dsh-koboldcpp-hands — give your DeepSeek Harness agent a pair of local hands.
English · 中文
A third-party tool plugin for DeepSeek Harness that lets the online model (your main conversation model) hand repetitive, token-cheap labor to a local KoboldCpp (llama.cpp) server — plain text work and vision work (image analysis / OCR / comparison).
The main model stays where your deployment puts it. When it decides a job is better done locally, it calls:
koboldcpp_run — run one prompt on the local text model (batch rewrites, name translations, string munging, short summarization, extraction).koboldcpp_vision — send images to a local multimodal model (OCR, image analysis, multi-image comparison) with structured report templates.
The plugin manages the local server lifecycle: it launches your KoboldCpp binary with your .kcpps launch config (which owns the GPU backend, model, mmproj, port), waits for the model to load, and stops the server on an idle timeout and/or harness exit. An externally started KoboldCpp is reused and never killed.
What it does
- Two model-facing tools registered on the harness tool registry (
ctx.tools), following the official dsh-tools contract (defineTool, canonical JSON values, pure render/presenters, exec.signal forwarding).
- On-demand server lifecycle: first tool call spawns
exePath with your kcppsPath + --port, polls /v1/models until healthy, self-heals if the server dies, and stops per stopBehavior (exit / idle / never). Windows process-tree termination (taskkill /T) because KoboldCpp relaunches itself as a child process.
- Text + vision wire support: non-streaming OpenAI-compatible chat-completions; images sent as the standard multimodal
content array.
- Three image sources for the vision tool: local file paths,
data:/http(s): URLs, or the images attached to the current conversation (read through the harness attachment service). Note: the conversation-attachment source requires a main model that declares image input — with a text-only main model only image_paths / image_urls work (see current limitation).
- Structured vision prompts structured, machine-verifiable report contracts:
analyze (8-section report), ocr (character-exact), compare (multi-image, 5-section) — plus a fidelity rule for the online model (relay verbatim, never invent, preserve uncertainty).
- Live config: a
llm-koboldcpp: section in the harness user-settings document overrides the plugin config without a restart; KOBOOLDCPP_EXE / KOBOOLDCPP_KCPPS env fallbacks.
- Safe ownership: external KoboldCpp processes are reused, never touched; only servers the plugin spawned are stopped.
What it does NOT do
- Does not replace the harness LLM provider — the online model stays the main model; the local model is only reached through the two tools.
- Does not decide GPU backends, model paths, or templates for you. Everything about the KoboldCpp launch lives in your
.kcpps file (backend usecuda/usevulkan/usecpu, model_param, mmproj, port). No probing, no auto-flags.
- Does not modify any DeepSeek Harness file; it is a pure add-on plugin.
- Does not bundle or host GGUF /
mmproj model files — bring your own.
- Does not use streaming or an API key (local server; no credentials involved).
- Does not run inside the harness process as a service — it spawns a separate KoboldCpp process only when needed.
Requirements
| Item | Requirement |
|---|
| Node.js | ≥ 20 |
| DeepSeek Harness | installed (npx @deepseek-ai/dsh web or a source checkout) |
| KoboldCpp binary | koboldcpp.exe (NVIDIA/CUDA) or koboldcpp-nocuda.exe (AMD/Vulkan), any release with /v1/chat/completions |
| GGUF model | your own; vision additionally needs a multimodal GGUF and its mmproj (set "mmproj" in the kcpps) |
Install
Inside your harness project (the directory whose cordis.yml / cordis.patch.yml composes your deployment):
npm install dsh-koboldcpp-hands
The package ships a dsh.bundle manifest, so the plugin row is inserted automatically by the harness loader — you only need the config overrides below. From a source checkout of the harness you can instead point the plugin row directly at a clone of this repo (plain-dependency install, row not auto-inserted):
- insert:
- id: koboldcpp-tool
name: '../dsh-koboldcpp-hands'
Configure
You own the launch settings. After an npm install the row already exists with defaults; override it by id from your profile cordis.patch.yml without insert (an insert with the same id crashes the loader with duplicate loader entry id):
- id: koboldcpp-tool
name: 'dsh-koboldcpp-hands'
config:
baseURL: 'http://127.0.0.1:5001' # must match the port in your kcpps
exePath: 'C:\path\to\koboldcpp-nocuda.exe' # your binary (CUDA or Vulkan build)
kcppsPath: 'C:\path\to\your-model.kcpps' # your launch config: backend + model + mmproj + port
autoStart: true
stopBehavior: idle
idleStopMinutes: 30
(For a plain-dependency install — git/local row, no bundle — use - insert: with the same row instead.)
The launched command is exactly:
koboldcpp-nocuda.exe "C:\path\to\your-model.kcpps" --port 5001
Full config reference (all 17 fields with defaults): docs/api.md §1.2.
Using the tools
koboldcpp_run — text
| param | type | required | meaning |
|---|
prompt | string | yes | instruction/text sent as a user message |
system | string | no | optional system instructions |
temperature | number | no | sampling temperature (0–2) |
max_tokens | integer | no | output cap (default maxTokens) |
stop | string[] | no | stop sequences |
Returns { text, reasoning?, model, usage, elapsedMs }.
koboldcpp_vision — images / OCR
| param | type | required | meaning |
|---|
mode | analyze/ocr/compare | no | built-in prompt template (default analyze) |
prompt | string | no | custom instruction (overrides the template) |
image_paths | string[] | no | local images (png/jpg/jpeg/webp/gif/bmp, ≤20 MB each) |
image_urls | string[] | no | data:image/... or http(s):// URLs |
temperature | number | no | sampling temperature (lower for OCR, ~0.2) |
max_tokens | integer | no | output cap |
stop | string[] | no | stop sequences |
Image sources resolve in order: explicit image_paths + image_urls → the most recent image(s) attached to the conversation → clear error. compare sends 2–4 images in ONE request for joint reasoning.
Returns { text, reasoning?, model, images, usage, elapsedMs }.
Vision requires a multimodal GGUF and its mmproj projector in the kcpps. Without mmproj the request completes but the model cannot see the image.
Current limitation: text-only main models — images arrive by link or local path only
In the current release (plugin 0.1.0, harness 0.1.0-rc.6), when your main model is text-only, koboldcpp_vision can only receive images through the two explicit channels: image_paths (local file paths) and image_urls (online / data: links). The conversation-attachment source is unavailable in that setup — and this is a hard limit of the harness, not of this plugin:
- When you paste or drop an image while a text-only model is selected, dsh rejects the message before it ever enters the session with
attachment-error / MODEL_DOES_NOT_SUPPORT_IMAGES (the UI shows "当前模型不支持图片，请切换支持图片的模型"). The check lives in dsh-host-apiproxy: the selected model's declared input modalities (from the pi-ai model catalog) must include image; a model catalogued as input: ["text"] (e.g. deepseek-v4-flash / deepseek-v4-pro on the opencode-go route) is refused.
- Even if an image part got through,
dsh-llm-pi-ai's streaming adapter rejects image content for the same text-only models (UNSUPPORTED_CONTENT), and subagent continuation sessions block images in the browser client entirely.
- Because the message is refused before it is durably attached, the "most recent conversation-attachment" source has nothing to read — unlike OpenCode / Pi, dsh does not currently turn a pasted image into a temporary file path for text-only models.
Workarounds that work today:
- Ask the model to call
koboldcpp_vision with image_paths: ["C:\\...\\photo.png"] — any path readable by the harness process.
- Or pass an online link:
image_urls: ["https://example.com/photo.png"] (also data: URLs).
- Or switch the main model to one whose catalog entry declares image input (e.g.
minimax-m3, qwen3.7-plus, kimi-k2.6, kimi-k3, grok-4.5 on the opencode-go route) — the conversation-attachment source then works automatically.
Tracked upstream in deepseek-harness discussion #1378 (request: allow image attachments for text-only models and deliver them to tools as links/paths). This section will be updated when the harness relaxes the restriction.
Roadmap
Possible / planned directions:
- More vision modes and prompt templates (document layouts, table extraction).
- Multi-model
autoswapmode support (kcpps-level; the wire model field is already configurable).
- Publishing to the npm registry and the
dsh-plugin topic.
- Batch jobs: drive many local calls from one agent turn.
Deliberately NOT planned:
- Automatic GPU/backend detection or flag injection — your kcpps is authoritative by design.
- Becoming an LLM provider adapter — the plugin stays a tool; the online model stays the main model.
- Streaming responses — tool calls get the full answer in one round trip (simpler and sufficient).
- Bundling model files (
gguf/mmproj) or modifying DeepSeek Harness itself.
Uninstall
Removing the plugin is as clean as installing it:
- Remove the plugin row from your profile
cordis.patch.yml (or cordis.yml):
# delete this block
- insert:
- id: koboldcpp-tool
name: 'dsh-koboldcpp-hands'
- Restart the harness (or let HMR reload if you edit the config live). The two tools (
koboldcpp_run, koboldcpp_vision) are unregistered automatically — the online model no longer sees them.
- Server lifecycle after removal:
stopBehavior: exit — the plugin stops its spawned KoboldCpp when the harness shuts down gracefully.stopBehavior: idle — the server stops after the idle window.stopBehavior: never — the server keeps running; stop it yourself (e.g. taskkill /PID <pid> /T /F on Windows).- An externally started KoboldCpp is never touched.
- No residue: the plugin writes nothing into the harness, leaves no lingering processes on graceful shutdown, and creates no config files of its own. If you installed it via npm, remove it with
npm uninstall dsh-koboldcpp-hands.
Removing the plugin package itself
- Installed via npm — one command removes the package from your project:
npm uninstall dsh-koboldcpp-hands
- Installed from a git clone (profile row points at the checkout) — remove the profile row, then delete the checkout:
Remove-Item -Recurse -Force C:\path\to\dsh-koboldcpp-hands
rm -rf /path/to/dsh-koboldcpp-hands
Version & compatibility
| Component | Version |
|---|
| This plugin | 0.1.0 |
| DeepSeek Harness | 0.1.0-rc series (tested against npm @deepseek-ai/* 0.1.0-rc.6) |
| Node.js | ≥ 20 |
| KoboldCpp | any release exposing /v1/chat/completions |
Peer dependencies (runtime): @deepseek-ai/cordis ^4.0.1, @deepseek-ai/dsh-tools/dsh-llm/dsh-session/dsh-attachment/dsh-settings/dsh-launch-environment >=0.1.0-rc.2, @deepseek-ai/schemastery ^3.18.1.
Development
npm install
npm run typecheck # tsc --noEmit
npm test # vitest run (45 tests: unit, tool, integration, Loader composition)
npm run build # tsc -> lib/
Tests include a REAL-composition tier (app boot → Cordis Loader → cordis.yml) per the harness testing policy, and a real-machine scenario driver (tests/real-driver.mjs) for autostart / reuse / not-running behaviors.
Documentation
Credits & thanks
- DeepSeek AI — the DeepSeek Harness platform this plugin plugs into, and the reference implementations (
dsh-llm-deepseek, dsh-tool-todo) that define the patterns we follow.
- LostRuins / KoboldCpp — the excellent local llama.cpp server with an OpenAI-compatible API that makes all of this possible.
- Cordis — the plugin runtime that powers the harness.
- The open-source models and quantizers (llama.cpp ecosystem, GGUF) that run locally on your machine.
License
MIT. Not affiliated with DeepSeek AI or LostRuins; dsh and koboldcpp are trademarks of their respective owners.
