dsh-backdoor-guard
English | 日本語
A DeepSeek Harness (dsh) plugin that stops trained-in backdoor payloads — model weights (e.g. a malicious QLoRa merge) that fire a specific tool call or credential exfiltration when a hidden trigger appears — before the tool executes.
Every model-requested tool call is priced by a dedicated decision model (Jev hosted, or a self-hosted LiquidAI d1 via the bundled shim — both speak the same Decision Index systemone schema). Calls judged unsafe are denied at tools/pre-execute and a SECURITY NOTICE lands in the transcript.
No fork, no logprobs, no regex rules to maintain: one public plugin seam + one judgment call (~100 ms hosted, ~30 ms local d1 on Apple silicon).
Why a decision model, not the agent's own LLM
- A backdoored model cannot be trusted to police itself; the judge is a different, tiny, frozen model.
- Decision models return a calibrated probability in one forward pass (zero output tokens) — fast enough to screen every tool call.
- Jev/d1 are built for exactly this: "yes/no with a probability" over a state document (routing, moderation, guardrails).
Install
dsh plugin add dsh-backdoor-guard # npm, once published
dsh plugin add github:miketako3/dsh-backdoor-guard # from GitHub
dsh plugin add file:/path/to/dsh-backdoor-guard # local checkout
The package ships a dsh.bundle layer, so plugin add wires the plugin row automatically.
Choose the judge
Option A: hosted Jev (TypeSafe AI)
export TYPESAFE_API_KEY=ts_... # from https://console.typesafe.ai
# profile cordis.patch.yml
- id: backdoor-guard
name: 'dsh-backdoor-guard'
config:
endpoint: https://api.typesafe.ai/v1/systemone
model: jev-latest
apiKeyEnv: TYPESAFE_API_KEY
Option B: local d1-3B (LiquidAI, open weights)
pip install "transformers>=5.14" torch torchvision pillow
python3 server/d1-server.py --model LiquidAI/d1-3B --port 8400
- id: backdoor-guard
name: 'dsh-backdoor-guard'
config:
endpoint: http://127.0.0.1:8400/v1/systemone
model: d1-3B
apiKeyEnv: '' # empty = no Authorization header (local, unauthenticated)
d1-3B is ~30 ms/decision on Apple M5 Pro, ~8 ms on RTX 4090, and reads 32k-token states — whole tool-call payloads fit comfortably. d1-omni-600M works too (text+audio; experimental).
How a call is judged
The judge receives a state document: the user's latest request, whether the agent visibly announced its action (silent invocations are the runtime signature of a memorized payload), the tool name, canonicalized arguments, and the turn's earlier calls. One Noul question prices legitimacy:
safe = the call serves the user's stated request; does not read/transmit unrelated credentials or secrets; no obfuscated / pipe-to-shell / self-decoding payloads; no unusual external endpoints; nothing that looks like a hidden-trigger action.
safe_probability >= threshold → the call proceeds. Below → deny before dispatch, with the probability in the deny reason.
Config reference
| key | default | meaning |
|---|
endpoint | https://api.typesafe.ai/v1/systemone | systemone-compatible judge endpoint |
model | jev-latest | judge model id |
apiKeyEnv | TYPESAFE_API_KEY | env var with the API key; '' = no auth (local) |
threshold | 0.5 | deny when safe probability falls below |
timeoutMs | 8000 | judge call timeout |
skipTools | [] | *-wildcard tool names never judged (e.g. ['read', 'web_*']) |
maxArgsChars | 2000 | argument cap shown to the judge |
maxRequestChars | 1000 | user-request cap shown to the judge |
onJudgeError | allow | allow = log + proceed when the judge is down; deny = fail closed |
Set onJudgeError: deny for high-security deployments; keep allow while tuning.
Threat model & limits
- Targets weight-borne backdoors that act through tools and credential exfiltration; pure text misbehavior is out of scope.
- The judge itself is trusted infrastructure: pin it (local d1 or your own Jev key), and do not route it through the model under review.
- Pair with network egress control for complete coverage.
Inspired by @pentacoxian_dev's rdsh-backdoor-guard research (entropy-based detection); this implementation deliberately uses a decision-model judge instead, so it needs no logprobs and works with any agent backend.
Develop
npm install
npm test # vitest: judge client + end-to-end plugin flow
npm run build # tsc → lib/
日本語
DeepSeek Harness（dsh）用プラグイン。モデルウェイトに焼き込まれたバックドア（特定トリガーで悪意あるツール呼び出し・認証情報窃取を発火）を、ツール実行前に判定モデルで止めます。
判定には専用の意思決定モデルを使います — ホスト型の Jev（TypeSafe AI）か、同梱 shim 経由のローカル d1-3B（LiquidAI 公開ウェイト）。どちらも同じ Decision Index（systemone）スキーマを話すので、endpoint を変えるだけで切り替わります。
- fork 不要・logprobs 不要・正規表現ルールの保守不要
- 判定は1回の順伝播（出力トークンゼロ）で済むので全ツールコールに挟める速度（ホストJev 約100ms、M5 Pro のローカル d1 約30ms）
- 判定対象: ユーザーの依頼文・前置きの有無（無言の発火は暗記ペイロードの特徴）・ツール名・引数・ターン内の先行呼び出し
- 閾値未満なら
tools/pre-execute で覆せない deny ＋ トランスクリプトに SECURITY NOTICE
インストール
dsh plugin add github:miketako3/dsh-backdoor-guard
judge の選び方
- ホスト Jev:
TYPESAFE_API_KEY を環境変数に（console.typesafe.ai で発行）
- ローカル d1-3B:
python3 server/d1-server.py --model LiquidAI/d1-3B を起動し、endpoint: http://127.0.0.1:8400/v1/systemone、apiKeyEnv: ''
脅威モデルと限界
- 対象: ツール経由で行動するウェイト由来バックドアと認証情報窃取。テキストだけの悪意ある出力は対象外
- judge は信用されたインフラであること（ローカル d1 か自分の Jev キー）。被審査モデルと同じモデルに判定させない
- 完全な防御にはネットワーク egress 制御と併用を
謝辞: @pentacoxian_dev 氏の rdsh-backdoor-guard 研究に着想を得た独立実装。エントロピー方式ではなく意思決定モデル方式を採用し、あらゆるエージェントバックエンドで動きます。
License
MIT
