dsh-prometheus
注重隐私的 Prometheus 指标和用于 DeepSeek Harness 的 Grafana 仪表板
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add dsh-prometheus@0.1.0Prometheus 以 dsh-prometheus 发布，当前版本为 0.1.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
点击下方 Badge 复制 Markdown，粘贴到 README 即可。
验证 package.json 声明的 GitHub 仓库，即可管理这个公开页面。认领后，Hub 会优先安排当前版本的安全扫描，并在通过后公开展示结果。
认领这个 Plugin →English | 中文
Prometheus metrics and a ready-to-import Grafana dashboard for DeepSeek Harness. The plugin observes public DSH lifecycle seams, keeps metric series bounded, and avoids exporting conversation or tool payloads.
Release status:
0.1.0is the first developer-preview release, tested against DSH0.1.0-rc.6.
The exact names, labels, and meanings are in docs/metrics.md.
| Component | Tested version |
|---|---|
| DeepSeek Harness npm packages | 0.1.0-rc.6 |
| Official source baseline | 47f943859bef60e4160492346772ded9b24f765a (2026-08-13) |
| Cordis | ^4.0.1 |
| Node.js | `^22.19.0 |
| prom-client | 15.1.3 |
DeepSeek Harness is a developer preview. Treat every DSH upgrade as a compatibility event and run the full test/release checklist.
Install the package into a DSH profile and verify its bundle layer:
dsh plugin --profile monitoring add dsh-prometheus
dsh --profile monitoring --dump-config
The bundle inserts one row with id prometheus. Verify the dump includes the dsh-prometheus layer before booting the profile.
The default mode: auto makes one activation-time decision:
webServer is already active and safely bound, register the exact /metrics route on it.127.0.0.1:9464.For the bundled Docker example, force standalone mode so Prometheus has a stable target. Add this row to the profile's cordis.patch.yml (later layers replace the whole config value):
- id: prometheus
config:
enabled: true
mode: standalone
host: 127.0.0.1
port: 9464
path: /metrics
allowRemote: false
maxLabelValues: 64
maxLabelValueLength: 80
| Field | Default | Description |
|---|---|---|
enabled | true | Disable all collectors and endpoint without removing the row. |
mode | auto | auto, webserver, or standalone. |
host | 127.0.0.1 | Standalone bind address: loopback or 0.0.0.0. |
port | 9464 | Standalone TCP port; 0 is intended only for tests. |
path | /metrics | Exact route, beginning with / and without a trailing slash. |
allowRemote | false | Required acknowledgement for 0.0.0.0 or a remote-facing WebServer route. |
maxLabelValues | 64 | Per-key dynamic-label vocabulary cap. |
maxLabelValueLength | 80 | Maximum accepted dynamic-label length. |
Remote exposure has no built-in authentication or TLS. Read SECURITY.md before enabling it.
curl --fail --silent http://127.0.0.1:9464/metrics
The endpoint accepts GET and HEAD, returns 405 for other methods, disables caching, and uses the Prometheus/OpenMetrics-compatible prom-client content type.
The repository includes a full local stack:
docker compose -f examples/docker-compose.yml up -d
Then open Prometheus at http://127.0.0.1:9090 and Grafana at http://127.0.0.1:3000 (admin / admin; change it outside local development). The datasource and DeepSeek Harness / Overview dashboard are provisioned automatically.
The standalone assets are examples/prometheus.yml and grafana/dsh-overview.json. Security, compatibility, and verification details live in docs/security.md, docs/compatibility.md, and docs/verification-report.md.
Useful queries:
# Turn throughput
sum(rate(dsh_agent_turns_total[5m]))
# Turn failure ratio
sum(rate(dsh_agent_turns_total{status=~"error|blocked|aborted"}[5m]))
/
clamp_min(sum(rate(dsh_agent_turns_total[5m])), 0.000001)
# P95 LLM latency by provider/model
histogram_quantile(0.95,
sum by (le, provider, model) (rate(dsh_llm_request_duration_seconds_bucket[5m])))
# Tool errors by tool
sum by (tool) (rate(dsh_tool_calls_total{status!="success"}[5m]))
See examples/prometheus/alerts.yml for starter alerts. Tune thresholds to the workload; quiet agent processes legitimately have zero throughput.
The collector never exports prompts, user/assistant messages, system prompts, tool arguments or results, error text, file paths, working directories, arbitrary metadata, or session/agent/call/job/subagent ids.
Only four runtime-controlled label keys exist: provider, model, tool, and job kind. Values must match [A-Za-z0-9][A-Za-z0-9_.:/-]*, remain within the configured length, and fit the per-key vocabulary cap. Rejected or excess values become __other__; the event is counted by dsh_metrics_label_overflow_total without logging the original value.
Provider/model/tool/kind names are still operational metadata. Do not put customer identifiers or secrets in those names.
All listeners, job observers, routes, sockets, and registries belong to the plugin's Cordis fiber. On hot reload or uninstall, the route is removed or the standalone server is closed and remaining sockets are destroyed. The registry is instance-local, so reload does not collide with process-global collectors.
Existing live sessions and jobs seed gauges only. Historical events are not replayed into counters. Counter resets across process restart/hot reload are normal Prometheus behavior.
# Upgrade after obtaining a newer tarball/version
dsh plugin --profile monitoring add ./dsh-prometheus-NEW.tgz
dsh --profile monitoring --dump-config
# Remove the dependency and bundle layer
dsh plugin --profile monitoring remove dsh-prometheus
Before a downgrade, compare metric and config contracts in CHANGELOG.md. Removing the plugin does not delete data already stored by Prometheus.
pnpm install --frozen-lockfile
pnpm check
pnpm test:coverage
pnpm pack
The suite covers config/cardinality, privacy, LLM/tool waterfalls, endpoint methods, remote-exposure gates, WebServer route teardown, standalone socket teardown, job/subagent lifecycles, and a real rc.6 AgentLoop turn. See CONTRIBUTING.md, docs/architecture.md, and docs/release-checklist.md.
See docs/troubleshooting.md and docs/known-limitations.md.
MIT