dsh-skillnotary
A DSH plugin that pins what your agent's skills are allowed to do — and tells
you when that changes.
An agent skill is a folder with a SKILL.md your agent loads and follows. It
usually contains runnable shell, scripts and MCP config: unreviewed code with
tool access, pinned to nothing, changing silently.
Most skill-security tools answer "is this skill dangerous right now?"
skillnotary answers "is this still the skill I approved — and is it still only
allowed to do what it did?" That is a different question, and it is the one
that catches the change that happens after your review.
dsh plugin --profile web add github:liyixuan201211/dsh-skillnotary
中文：安装、更新、审查 agent skill 或 DSH 插件之前，先锁定它、审计它的能力，并在它被上游改动时发现。多数工具回答"这个 skill 现在危不危险"，本插件回答的是"它还是我批准的那个吗，还只被允许做当初那件事吗"。
What it changes
Once installed, the agent follows skills/skillnotary/SKILL.md: before adding or
updating a skill it runs skillnotary audit (what can it do?), verify (is it
what was approved?) and policy (is it allowed here?) — and reports a
critical/high finding instead of quietly installing.
It ships two skills:
skills/skillnotary/ — the review loop above, for the moment you adopt or
update something.skills/skillnotary-drift-watch/ — for after that moment: it sets up a
recurring verify (via DSH's scheduling tools) so a skill that changes once you
have stopped looking is noticed, instead of being discovered later.
The moment it exists for:
$ skillnotary verify
✗ 2 drift(s) detected:
✗ release-notes [integrity-changed] content digest changed: e4qHXFFicS -> Cj7IuwNp3Gb
✗ release-notes [capabilities-changed] capabilities changed: [exec] -> [agent.spawn,exec]
A hash tells you "something changed". That second line tells you the skill gained
the ability to spawn further agents. Reproduce it yourself offline:
git clone https://github.com/liyixuan201211/skillnotary
cd skillnotary && bash demo/run.sh
Install
# this plugin
dsh plugin --profile web add github:liyixuan201211/dsh-skillnotary
# the tool it drives (not on npm yet; run it from git today)
npx --yes github:liyixuan201211/skillnotary --help
Pin the commit if you prefer:
dsh plugin --profile web add github:liyixuan201211/dsh-skillnotary#<commit>
This plugin ships no boot-time code
cordis.patch.yml is an empty patch, deliberately:
- a boot-time
insert would run code inside the DSH process, for every profile,
with your full privileges — a security plugin that widens the boot graph is
itself attack surface;
- a bad insert can take down the whole boot layer (
~/.dsh/cordis.patch.yml
carries an explicit warning about this), and shipping something that could do
it would be careless;
- the skill needs no privileges — it asks the agent to run
npx skillnotary … through the normal shell tool, where you can see and
approve each command.
So if you are reviewing this plugin before installing it: cordis.patch.yml is
the whole story, and nothing there executes.
It audits itself
Both skills are checked by skillnotary like any other skill, and both declare
allowed-tools: Bash because they do ask the agent to run commands. Two findings
survive, and we left them in rather than gaming them:
skillnotary MEDIUM R007 Runtime package installation (SKILL.md:45)
> npx --yes github:liyixuan201211/skillnotary --help
skillnotary-drift-watch MEDIUM R007 Runtime package installation (SKILL.md:23)
> sn() { npx --yes github:liyixuan201211/skillnotary "$@"; }
· 4 finding(s) suppressed (config.ignore matched R001:reference/*; config.ignore matched R007:reference/*)
✓ no findings at or above "high"
npx is runtime package installation. Suppressing it would have been the
dishonest move, and medium is below the default maxSeverity: high, so the CI
gate still passes.
The suppressed findings are the same shape of false positive, in the one place it
is unavoidable: reference/capabilities.md and reference/workflow.md quote
commands in order to document them, and a capability table has to name the strings
the detector looks for. So skillnotary.config.json exempts reference/* — by
rule and by path, never an entire file:
{ "version": 1,
"ignore": ["R001:reference/*", "R007:reference/*"],
"allowInlineSuppressions": false }
The scoping is the point: R001 is silenced only inside reference/, so a
capability the entry-point SKILL.md genuinely exercises is still reported.
What skillnotary does, in one table
| Pillar | Command | What you get |
|---|
| Lock | lock | skills.lock: content digest, resolved commit, and the capability set |
| Verify | verify | bytes and capabilities drifting from what you approved |
| Attest | keygen / sign | ed25519 over a DSSE envelope with an in-toto statement |
| Govern | policy | per-skill allow/deny, per-capability gates, severity threshold |
| Repair | fix | write the allowed-tools a skill actually needs into its SKILL.md |
| Install | apply | copy locked skills into a harness dir, re-checking every digest |
| Gate | ci | all of the above as one CI step |
Zero runtime dependencies; TypeScript with no build step (Node ≥ 23.6).
Honest limits
- Detection is context-scoped regex, not a sandbox. Targeted obfuscation
evades it; a clean report means "no known signals", not "safe".
- Only the head of each file is scanned (1 MB per file, 64 MB per skill); rule
R025 says so when that truncates instead of hiding it.
- Attestations are DSSE + in-toto, but this is not Sigstore — no keyless/OIDC,
no transparency log.
- The lockfile pins a skill; it does not sandbox what the skill does when your
agent runs it.
Links
Licence
MIT
