效果评测（dsh-verdict）
中文名称：效果评测；英文名称：Verdict。用于评估规则、技能或模型调整前后的效果，使用可重复的评测与证据判断是否改善。包名 dsh-verdict、命令 verdict 保持不变。
Measure whether a change to your dsh setup actually helped.
Register repeatable cases, run them, and diff the results before and after you change
rules, skills, model settings, or plugins. A dsh setup change usually feels like an
improvement; this plugin is what tells you whether it was one.
Status
Published to npm as dsh-verdict@0.1.0.
Implemented: capture, durable store, rule promotion into AGENTS.md, guard compilation,
automatic re-verification (also scheduled every six hours in a resident host), skill
distillation, and the settings page. Covered by 87 unit tests against a real filesystem
(npm test), including static renders of the panel. CI runs on Ubuntu and Windows.
Verified on real hardware: the plugin loads inside dsh 0.1.5-rc.2, the host route
GET /api/verdict.panel serves the panel payload, and the browser half registers a
"Verdict" page in Settings without errors.
What it does
ctx.verdict.learn('never commit the file secrets.env')
// -> rule captured, guard compiled for this platform, written into AGENTS.md
// behind <!-- verdict:<id> -->, and now re-checked on demand
ctx.verdict.check()
// -> { passed, violated, broken, exitCode }
// exitCode is 1 only for real violations — a guard that cannot run is
// reported separately and never turns a pipeline red
A rule is only promoted if it has an executable check. A rule whose check stops holding
is reported; a rule that keeps failing is demoted to ineffective after three distinct
violation episodes — not three runs, so a long-lived failure cannot inflate the count.
The compiler knows a few sentence shapes — file existence, forbidden files, forbidden
root globs ("no *.pem files"), and required file contents ("the file README.md must
mention install") — and refuses any sentence it cannot express faithfully on the current
platform, leaving it a candidate rather than emitting an approximate command.
Install
From npm (recommended — the tarball ships pre-built lib/):
dsh plugin add dsh-verdict --profile web
Or pin the exact version:
dsh plugin add dsh-verdict@0.1.0 --profile web
For local development without publishing:
dsh plugin add link:/path/to/dsh-verdict --profile web
The plugin ships a cordis.patch.yml, so the profile registers it on install — there is
nothing to insert by hand. GitHub source installs will not work directly because lib/
is gitignored and dsh forbids install-time build scripts; use npm or link: instead.
CLI
The same modules are reachable without booting dsh, so the plugin's claims can be
checked independently and check can run in CI:
verdict learn "never commit the file .env" # capture + compile a guard + promote
verdict check # exit 1 on real violations, 0 otherwise
verdict list
verdict recall "env"
verdict session --intent "verify a change" --step "run tests" --step "commit"
verdict skills --write
verdict panel # the panel's view of the project
verdict diff before ; verdict diff after ; verdict diff
# did that change help? answered from two snapshots
verdict doctor # check this package against dsh's plugin rules
verdict inventory --audit # installed plugins, and what would stop them loading
verdict check is the CI-relevant one: it exits non-zero for a genuine violation and
stays zero when the only problem is a guard that cannot run, so a typo in a guard can
never turn a pipeline red on its own.
verdict doctor covers everything about loading that is static — manifest, build output,
patch id agreement, peer placement, lifecycle scripts — so the one thing left for a real
dsh install is the loader itself.
Web panel
The plugin also ships a browser half (exports["./client"]). It registers a
settings.section slot — a "Verdict" page inside Settings — and reads the panel
from GET /api/verdict.panel, which the host half registers on the web
connection when one exists. Hosts without a web client simply skip it.
The page draws only what the data supports: a rule shows "guard passed" because
a guard ran and passed, never because it was promoted, and a before/after
comparison without both snapshots states that instead of an improvement. Skill
candidates distilled from repeated successes are listed with a write action
(POST /api/verdict.skills.write); the response carries whether the written
skill's guard actually runs.
scripts/bundle-client.mjs replicates the loader's lazy-CJS factory artifact
(banner, intro vars, footer, react left external to the platform module
table). The bundle is lib/verdict.web.js — deliberately not lib/client.js,
which is the compiled host module src/client.ts.
Layout
| Path | Role |
|---|
src/index.ts | Host half — apply(ctx), exposes the verdict service |
src/identity.ts | Rule id derived from text, so repeats collapse into one rule |
src/store.ts | Durable .verdict/ledger.json; atomic writes |
src/guard.ts | Rule → shell command; run it; classify pass / violated / broken |
src/promote.ts | Write rule into AGENTS.md behind a marker; verify its presence |
src/check.ts | Verification pass, episode-based recurrence, exit code |
src/pipeline.ts | learn: capture → compile → promote |
src/session.ts | Append-only session log; what actually happened |
src/skills.ts | Distil repeated successes into a skill (M6) |
src/cli.ts / src/bin.ts | Command-line entry, usable without dsh |
src/routes.ts | GET /api/verdict.panel and POST /api/verdict.skills.write on the host's web connection |
src/verdict-view.ts | Payload → view model, shared by host route and browser half |
src/schedule.ts | The six-hour re-check report line for resident hosts |
src/client/index.tsx | Browser half — panel state, fetch, registration |
src/client/view.tsx | Browser half — pure rendering, statically tested in Node |
scripts/bundle-client.mjs | esbuild bundle in the loader's factory format |
cordis.patch.yml | Profile registration (insert id dsh-verdict) |
package.json | Package manifest; host packages stay in peerDependencies |
Payloads must be peers
dsh reserves certain packages for the host. Copying one into this plugin's own
dependencies produces a nested or aliased copy, which the profile validation rejects.
So @deepseek-ai/cordis and every @deepseek-ai/dsh-* package this plugin uses belong in
peerDependencies (plus devDependencies for local typechecking), never in
dependencies.
No lifecycle scripts
Desktop rejects plugins whose dependencies carry install lifecycle scripts unless the
package is in the desktop project's reviewed allowBuilds policy. This plugin therefore
does no work at install time — everything happens in apply.
Develop
npm ci
npm run typecheck
npm run build
npm test
node tests/smoke.mjs # end-to-end walkthrough against temp dirs
