@yadsh/dsh-kv-persist
dsh-kv-persist keeps local LLM sessions warm across session switches and restarts.
It maps DeepSeek Harness sessions to persistent inference-cache snapshots and
restores them when a session becomes active again. The initial backend uses
llama.cpp's slot save/restore API, allowing large agent contexts to resume
without repeating a full prompt prefill.
KV state is treated strictly as an optimization: DSH's session log remains the
source of truth, and any missing, stale, or incompatible cache automatically
falls back to normal inference. The model never knows this plugin exists.
Features
- Maps DSH sessions (
sessionId + provider + model) to llama.cpp slot
snapshots via GET /slots and POST /slots/{id}?action=save|restore|erase.
- Lazy restore: nothing is loaded until the session's first managed request;
resident sessions run with zero disk I/O and zero management calls.
- Save-before-switch, idle checkpoints, and shutdown/flush checkpoints, with
dirty generations and save coalescing instead of a naive boolean.
- Runtime compatibility gate: snapshots restored only when the server
instance, provider, model, and
runtimeKey generation match; the rest are
marked invalid and skipped, never deleted.
- Cold fallback everywhere: persistence failures degrade to ordinary
inference; optional strict mode is the only failure path.
- Circuit breaker stops a broken backend from costing latency on every
request.
- Auxiliary requests (session titles, compaction) are coordinated so they
never pollute or hijack a session's slot ownership.
- Structured
kv.* logging, in-memory metrics, status() and doctor()
diagnostics on the service.
Requirements
- DeepSeek Harness >= 0.1.5-rc.2 < 0.2.0
- A llama.cpp
llama-server started with --slots --slot-save-path <dir>
and --parallel 1 for the single-slot mode (see the design doc §6).
The server flag does not replace the plugin's local lease, which also
serializes save, restore, erase, and terminal stream bookkeeping.
- Node.js >= 22
Installation
Install the published npm package by name:
dsh plugin --profile <profile> add @yadsh/dsh-kv-persist
From sources, build the package and link the checkout:
pnpm nx run @yadsh/dsh-kv-persist:build
dsh plugin --profile <profile> add ./plugins/dsh-kv-persist
Configuration
Configure the plugin under the kv-persist key in the DSH profile. Only
providers listed under providers are ever coordinated.
| Option | Type | Default | Description |
|---|
enabled | boolean | true | Master switch; false passes everything through. |
backend.baseURL | string | http://127.0.0.1:8080 | llama-server management endpoint. |
backend.apiKey | string | "" | Bearer key for the management API; never logged. |
backend.requestTimeoutMs | number | 15000 | Bounded timeout per persistence call. |
providers | string[] | [] | Managed provider routes; empty = plugin inert. |
mode | "single-slot" | "single-slot" | v0.1 supports single-slot only. |
slotId | number | 0 | Physical slot used in single-slot mode. |
runtimeKey | string | "" | Runtime identity escape hatch; changing it hides old snapshots (they are kept, not deleted). |
checkpoint.onSwitch | boolean | true | Save a dirty slot before reassignment. |
checkpoint.onShutdown | boolean | true | Final checkpoint on plugin disposal. |
checkpoint.onSessionFlush | boolean | true | Checkpoint on the session flush event. |
checkpoint.idleMs | number | 30000 | Idle checkpoint delay; 0 disables. |
checkpoint.onTurnEnd | boolean | false | Checkpoint after every user turn. |
checkpoint.onStepEnd | boolean | false | Reserved for per-step checkpoints. |
|
Full configuration rationale: design doc §35.
Compatibility
- DeepSeek Harness
>=0.1.5-rc.2 <0.2.0 (see compatibility.json)
- Node.js >= 22
- llama.cpp server with the slots management API enabled
Development
pnpm nx run @yadsh/dsh-kv-persist:lint
pnpm nx run @yadsh/dsh-kv-persist:typecheck
pnpm nx run @yadsh/dsh-kv-persist:test
pnpm nx run @yadsh/dsh-kv-persist:build
pnpm nx run @yadsh/dsh-kv-persist:verify
Optional end-to-end check against a live server (no GPU in CI):
DSH_KV_TEST_LLAMA_URL=http://127.0.0.1:8080 pnpm --filter @yadsh/dsh-kv-persist test:llama
See SPEC.md for the product contract and implementation status.
License
MIT
