dsh-model-deploy
面向真实硬件目标的本地或远程 AI 模型部署基准测试
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add github:ethanrise/dsh-model-deploy#4f1f4e7430c9ced7f236f3a6ffaa73716fd0b4dfModel Deploy 以 dsh-model-deploy 发布，当前版本为 0.1.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
Local or remote AI model deployment benchmarking for real hardware targets.
dsh-model-deploy helps answer a practical deployment question: will this model meet my requirements on the machine that will actually run it?
V0.1 focuses on ONNX + ONNX Runtime, with local execution on Windows/Linux and agentless remote execution over SSH to POSIX targets such as Linux workstations and Jetson devices.
The plugin runs its benchmarks in Python. Any interpreter with these packages works:
pip install numpy onnx onnxruntime psutil
For CPU inference, standard onnxruntime is enough. GPU execution requires an ONNX Runtime build/provider compatible with the machine (e.g. onnxruntime-gpu with matching CUDA/cuDNN libraries); if the provider fails to load, results say so rather than reporting CPU numbers as GPU numbers.
The plugin picks the interpreter automatically: DSH_MODEL_DEPLOY_PYTHON if set, then python3/python, then conda/miniforge environments under your home directory — the first one that can import numpy, onnx and onnxruntime.
In DeepSeek Harness, open Plugins → Add plugin and enter:
https://github.com/ethanrise/dsh-model-deploy
(or a local checkout path). This registers the model-deploy MCP server and the model-deploy skill; no npm link or PATH setup is needed. Restart DSH after installing or updating.
git clone https://github.com/ethanrise/dsh-model-deploy.git
cd dsh-model-deploy
pip install -e ".[runtime]"
npm install && npm run build # only needed after changing src/
node lib/doctor.js # shows which interpreter was chosen and checks dependencies
Inspect a model:
dsh-model-deploy inspect model.onnx
Probe the current machine:
dsh-model-deploy env
Benchmark locally:
dsh-model-deploy bench model.onnx --runs 100
Benchmark with deployment constraints:
dsh-model-deploy bench model.onnx \
--min-fps 30 \
--max-p95-ms 35 \
--max-model-mb 200
Benchmark a real remote target using an existing SSH alias:
dsh-model-deploy remote-bench orin-dev model.onnx --runs 100
The remote target currently needs python3, numpy, and onnxruntime already installed. V0.1 deliberately detects/uses the environment rather than modifying it.
model_inspectdeployment_environmentbenchmark_localssh_preflightbenchmark_remote_sshModel paths must be absolute (or start with ~/).
benchmark_local and benchmark_remote_ssh accept inputShapes (e.g. {"images": [1, 3, 640, 640]}) and defaultDynamicDim for dynamic-input models, plus the gate arguments minFps, maxP95Ms, maxModelMb and requireProvider.
Benchmark results report the execution provider the ONNX Runtime session actually used. When ORT silently falls back (e.g. CUDA listed as available but its libraries fail to load), provider_fallback is true and fallback_reason carries ORT's own error. runtime records the Python interpreter, ONNX Runtime and NumPy versions that produced the numbers.
The Python interpreter is chosen from DSH_MODEL_DEPLOY_PYTHON, then python3/python, then conda/miniforge environments under the home directory — the first that can import numpy, onnx and onnxruntime. DSH's MCP client scrubs inherited DSH_* variables, so to pin an interpreter under DSH set it in the patch entry's env.
SSH credentials are not passed to the model. Use ~/.ssh/config, ssh-agent, or the operating system's normal SSH credential handling.
TensorRT engine building, FP16/INT8 conversion/calibration, Docker/WSL executors, OpenVINO, TFLite, RKNN and NCNN are intentionally deferred. The next major backend target is TensorRT.
The project distinguishes:
It does not invent latency numbers for hardware that was not tested.
MIT