dsh-stability-audit (plugin stability audit)
Scan installed DeepSeek Harness (dsh) plugins and get a stability risk grade + fix suggestion before they break your harness — static analysis, optional isolated install verification, and machine-readable output for any agent to surface problems and remedies fast.
Built from real incidents: every grading rule comes from actual dsh ecosystem failures — barricade's hooks killing the whole tool chain, codegraph freezing the event loop at startup, trailmap's version conflict breaking the tool layer, dsh-tools single-instance corruption, doc-guard/secret-guard missing-inject boot crashes, 7 unbuilt plugins crashing on load, memento/taskboard undeclared deps (Cannot find package)... The rules are the troubleshooting manual, codified.
Community plugins are often young projects. This plugin answers one question: will installing/keeping this plugin hurt my dsh? Each installed plugin gets a 🔴/🟡/🟢 grade, evidence, and a 🔧 fix suggestion, so you decide on facts, not vibes.
Why
- One plugin can kill the entire tool layer (hook surface — barricade incident; dsh-tools single-instance — trailmap incident)
- Startup scans can freeze the event loop for minutes on big workspaces (codegraph incident)
- Bad plugin config makes dsh silently half-activate plugins (preflight failure)
- @deepseek-ai/* version conflicts can kill the whole tool pipeline
- Every failure period pollutes active sessions (dangling tool-calls) that need cleanup after recovery
These are the real failure modes this plugin statically detects — before you step on them.
Install
# Option 1: npm (recommended — prebuilt, skips allowBuilds approval)
dsh plugin --profile web add dsh-stability-audit
# Option 2: GitHub source
dsh plugin --profile web add github:chunfenxiazhi-collab/dsh-stability-audit
After restarting dsh web, agents can call the stability_audit tool (just say "run the plugin stability audit"); or use the CLI:
git clone https://github.com/chunfenxiazhi-collab/dsh-stability-audit.git
cd dsh-stability-audit && npm test
node cli.mjs # scan the real web profile
node cli.mjs --dynamic # also run isolated install verification (~2s/plugin)
node cli.mjs --json # machine-readable output (for other agents)
node cli.mjs --remote owner/repo # remote plugin static pre-audit (clone to temp dir, no install)
node cli.mjs --remote owner/repo --dynamic # full remote pre-audit (clone + isolated install smoke)
node cli.mjs --remote owner/repo --json # remote pre-audit JSON output
Grading dimensions
| Signal | Grade | Case |
|---|
| Tool hook / middleware registration (pre/post-execute) | red | can intercept the whole tool chain (barricade) |
| Top-level sync scan / indexing at startup | red | blocks the event loop (codegraph) |
| Load-time sync service access missing from inject | red | boot crash (doc-guard/barricade incidents) |
| main points to unbuilt source (TS/missing file) | red | load-time crash (7 unbuilt plugins) |
| Preflight report critical | red | flagged by dsh at boot |
| setInterval polling | yellow | normal for update checks; watch frequency |
| >=10 global event listeners | yellow | every broadcast event gets slower |
| No dsh.bundle manifest | yellow | installed but never activated (research-plugins) |
| >50 dependencies | yellow | dependency-tree pollution risk |
| Non-build install scripts (prepare etc.) | yellow | npm lifecycle runs arbitrary code |
| Deferred service access missing from inject | yellow | may be undefined at runtime (cron-style) |
| @deepseek-ai/* range excludes host synced pkg version | yellow | pnpm hoists old build -> tools dead (trailmap) |
| Isolated install + boot smoke | pass/fail | temp DSH_HOME, dsh plugin add, headless boot, no loader errors |
| Entry imports undeclared deps (missing-dep) | red | Cannot find package after install (taskboard/memento) |
| No main and no candidate entry (no-entry) | red | loader fails importing by package name (verify) |
| Data exfiltration (process.env via fetch/WebSocket) | red | OpenClaw 341 malicious-skill top pattern |
| Hardcoded remote endpoints (URL/WebSocket) | yellow | for human review (normal API calls common) |
Every hit carries a 🔧 fix suggestion (executable command or manual action) that can be handed to the user or another agent.
Batch testing: --remote accepts multiple specs (comma- or space-separated) to pre-audit a batch of online plugins in one run (clone → static → isolated install smoke, host untouched). Verified sample: docs/audit-report.md (18 plugins, post-verification verdicts).
How it works
- Collect plugins from ~/.dsh/profiles//package.json (node_modules + plugin-src)
- Static scan: entry file + package.json signals (regex-based, zero dependencies, no AST)
- Optional dynamic check (lib/dyncheck.js): temp DSH_HOME -> isolated dsh plugin add -> headless boot -> loader-error scan -> cleanup
- Optional remote pre-audit (lib/remote.js): git shallow clone to temp dir -> static rules -> (optional) isolated install smoke -> cleanup
- Render report: Markdown (human) or JSON (
--json, schema dsh-stability-audit/v1, agent-readable)
Principles: read-only, never runs the audited plugin, zero side effects. Grades are suggestions for human confirmation, not verdicts.
Architecture diagram
dsh-stability-audit runtime architecture
The runtime architecture is drawn with dsh-archify (MIT — a dsh port of tt-a1i/archify, also MIT, itself derived from Cocoon AI's "architecture-diagram-generator"). The MIT notice is embedded in the rendered HTML head; spec and evaluation notes are kept in-repo:
JSON output (for agents)
node cli.mjs --json
Per plugin: name / version / grade / source / dynamic / findings[], each finding carries ruleId / severity / desc / evidence / fix (remedy). Other agents can act on it directly.
Known limitations (honest list)
- Static analysis cannot measure runtime behavior (hook throws, real stall duration) — red means "needs a look", not "definitely broken"
- Isolated boot uses a headless profile: plugins depending on web-only services (e.g. storageDomain) show ❌ in isolation but may be fine in web — treat as an environment-dependency hint
- Runtime probes (event-loop latency, hook timing) are on the v2 roadmap
- Fix suggestions are hints, not auto-applied: environments differ (junction/overrides fixes depend on the specific mechanism), agents should confirm before executing
Isolated test vs real environment (what it can and cannot catch)
Isolated testing (temp DSH_HOME + headless smoke) answers: "can this plugin install on its own and boot cleanly?" — filtering out ~80% of the failure modes (unbuilt entry, missing inject, dependency conflicts, load crashes).
| Dimension | Isolated env | Real web profile | Impact |
|---|
| Services | headless boot | web services + all host services | plugins depending on webServer/webRuntime stay pending in isolation; real behavior not observable |
| Co-installed plugins | only the audited one | many plugins interacting | inter-plugin conflicts not caught (hooks stepping on each other, service overrides) |
| Config | empty | real config (API keys, models, paths) | config-dependent plugin paths untested |
| Credentials/network | no API key | model APIs available | LLM-dependent features only smoke, never execute |
| Data | empty storage | real sessions/libraries | data-migration plugins untested |
| Runtime duration | seconds of boot smoke | days of residency | setInterval leaks, memory growth not caught |
| Permissions/build | same machine, same permissions | same machine, same permissions | ✅ identical |
Suggested usage: use isolated testing as bulk screening — exclude red/fail targets; for plugins you actually want, install manually and observe. Automatically testing plugins against the real environment is not recommended — plugins have real destructive power (see the dsh-troubleshooting incidents).
Releases
License
MIT (c) 2026 chunfenxiazhi
Contributing
Issues and PRs welcome. Tests: npm test (node:test, zero dependencies).
Commit convention: English subject line + optional Chinese body (e.g. feat: remote plugin pre-audit).
