dsh-local-models
dsh 插件：一个 Local Models 设置选项卡，用于启动和停止与 dsh 宿主进程共存的 llama-server 子进程。
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add github:Vmarcelo49/dsh-local-models#42cd4c64b6a44229c865b33ddbb4579992a5328cLocal Models 以 dsh-local-models 发布，当前版本为 0.2.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
A dsh addon that adds a Local Models tab to the dsh Web GUI: pick a .gguf file, tune context and speculative decoding, watch a live VRAM estimate, and load it through llama-server — then register the running server as an LLM provider in dsh with one click.
Built against stock upstream llama.cpp (llama-server). No fork, no patches, no build step: the client bundle is hand-written React.createElement (no JSX toolchain) and the node half is dependency-free.
.gguf only) with a header-only GGUF parse (architecture, quant, layers, context length, MoE detection) behind POST /local-models/gguf-metaoff/low/medium/xhigh), optional vision mmproj (GPU or CPU offload), MoE expert placement (--cpu-moe / --n-cpu-moe / top-k override) with a fit-to-VRAM helper--models-preset); models load on demand, one resident at a time by default. Starting the router automatically (re-)registers its models in dsh — no manual Register press.llm-pi-ai provider route (vision modality + thinking levels included)llama-server log from the tabdsh with the web profile (the plugin composes into it)llama-server binary (upstream llama.cpp, Vulkan/CUDA/CPU — whatever your machine uses)lib/client.js (TOTAL_VRAM_BYTES, SAFE_MARGIN_BYTES) if yours differscd ~/.dsh/profiles/web
dsh plugin --profile web add /path/to/dsh-local-models
# then add "dsh-local-models" to the "bundles" array in package.json
Restart the dsh web process (bundle composition picks up only at boot), refresh the browser, open Settings → Local Models.
Node-half changes (routes, inject list) need a dsh restart; client-half changes only need a page refresh.
local-<alias>) appears in the Models picker.| Variable | Default | Meaning |
|---|---|---|
LOCAL_MODELS_PORT | 8080 | llama-server port |
LOCAL_MODELS_BIN | ~/Projetos/llama.cpp/build/bin/llama-server | server binary |
LOCAL_MODELS_SHORTCUTS | /mnt/raid0/GGUF | colon-separated file-browser shortcut dirs (name=path for custom labels) |
LOCAL_MODELS_MMPROJ_CPU | 1 | vision projector weights in RAM (0 = offload to GPU) |
LOCAL_MODELS_ROUTER_MAX | 1 | max simultaneously resident router models |
LOCAL_MODELS_MAX_IMAGE_BYTES | 10485760 | vision image guard |
LOCAL_MODELS_IMAGE_PIXEL_BUDGET | 4194304 | vision pixel budget |
DSH_HOME | ~/.dsh | data dir (local-models/profiles.json, llama-server.log) |
Launch flags are fixed to the validated daily config: full offload, -b 2048 -ub 512 -t 4 -np 1, --flash-attn on --kv-unified, --cache-type-k q8_0 --cache-type-v q4_0, MTP --spec-type draft-mtp --spec-draft-n-max N --spec-draft-p-min 0.75 (dropped above 131072 ctx unless the profile sets ignoreCtxCap — the tab's “ignore the MTP ctx softcap” checkbox, which forces the draft on at any ctx and may OOM or collapse decode).
/local-models)| Route | Meaning |
|---|---|
GET /local-models/browse?dir= | dirs + .gguf files |
POST /local-models/gguf-meta | {path} → parsed GGUF header (cached) |
GET /local-models/status | state + fresh /health probe |
GET /local-models/logs?offset=&max= | incremental tail of llama-server.log |
POST /local-models/run | spawn the server |
POST /local-models/stop | stop the child (or reap the port) |
POST /local-models/profiles / GET | save (upsert) / list profiles |
POST /local-models/profiles/remove | delete a profile |
POST /local-models/router/start | build presets from profiles + start router |
POST /local-models/router/unload | unload one router model |
POST /local-models/router/unload-all | unload all router models |
POST /local-models/register | add the ready server as an llm-pi-ai route |
lib/index.js node half: process manager, GGUF parser, routes, presets
lib/client.js browser half: settings tab (single build-free bundle)
skills/ operator skill: spawn-parity checklist, profile audits
docs/ UI mockup
Pure, exported helpers (normalizeEffort, moeArgsFor, generateRouterPresets, buildProviderProfile, profiles store) are unit-testable without a running server; node lib/index.js /path/to/model.gguf dumps a parsed header as a self-test.
See KNOWN_ISSUES.md — most notably, the VRAM estimate is approximate for Gemma-family layouts.
MIT — see LICENSE.