dsh-tool-chaos
面向 DeepSeek Harness 工具流水线的确定性故障注入、自主韧性实验和机器可读证据
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add github:cyanseek/dsh-tool-chaos#d194c0fa3a071d7bf3e941e14e1c33cb497e938bTool Chaos 以 dsh-tool-chaos 发布，当前版本为 0.1.0-next.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
English · 简体中文
Deterministic fault injection and evidence-backed resilience tests for the DeepSeek Harness tool pipeline.
dsh-tool-chaos runs an isolated DSH experiment through baseline → dry-run → fault injection → PASS / FAIL / INCONCLUSIVE. It helps plugin and Agent authors verify retry, fallback, cancellation, policy-denial, and untrusted-result behavior before those paths fail unexpectedly.
Status: 0.1.0-next.0 prerelease. DSH is still in developer preview, so pin versions and review the tested compatibility matrix.
deny, error, delay, abort, and block.The npm prerelease has not been published yet. The currently available path is a source checkout and a locally built tarball:
git clone https://github.com/cyanseek/dsh-tool-chaos.git
cd dsh-tool-chaos
npm ci
npm run build
npm pack --silent
dsh plugin --profile chaos-demo add ./dsh-tool-chaos-0.1.0-next.0.tgz
dsh --profile chaos-demo --dump-config
The effective config should contain tool-chaos and tool-chaos-reporter. Both are inert after installation: the main plugin is disabled and the reporter is disabled.
Generate a complete dry-run overlay:
node bin/dsh-tool-chaos.mjs init .dsh-chaos/retry.patch.yml --preset retry
Review the generated tool matcher, task, and assertion, then run it against the disposable profile:
node bin/dsh-tool-chaos.mjs run \
--mode dry-run \
--profile chaos-demo \
--patch .dsh-chaos/retry.patch.yml \
--task "Use web_fetch and recover from one transient failure" \
--expect "expected stable phrase" \
--no-install \
--json
A dry-run must show the intended dryRun: true decision while leaving tool execution unchanged. Run a real experiment only after that match is exact.
DSH can install this repository directly:
dsh plugin --profile chaos-demo add \
github:cyanseek/dsh-tool-chaos#<commit-sha>
pnpm 10+ can intentionally block a Git dependency's prepare script. If it does, review the pinned source and copy the exact package key printed by pnpm into that profile's pnpm-workspace.yaml:
allowBuilds:
dsh-tool-chaos@https://codeload.github.com/cyanseek/dsh-tool-chaos/tar.gz/<commit-sha>: true
Rerun the install and verify the effective config. This approval allows package code to execute during installation; a reviewed tarball remains the simpler option.
To uninstall and verify removal:
dsh plugin --profile chaos-demo remove dsh-tool-chaos
dsh --profile chaos-demo --dump-config
Installed defaults cannot inject a fault:
- id: tool-chaos
config:
enabled: false
dryRun: true
seed: dsh-tool-chaos
allowGlobalWildcard: false
rules: []
A bounded preview rule looks like this:
- id: tool-chaos
config:
enabled: true
dryRun: true
seed: retry-suite-v1
allowGlobalWildcard: false
rules:
- id: first-github-error
tool: github_*
action: error
scope: root
argumentsPattern: ''
probability: 1
afterMatches: 0
every: 1
maxInjections: 1
delayMs: 1000
message: deterministic GitHub transport failure
tool is an anchored, case-sensitive glob. argumentsPattern runs against stable key-sorted JSON. DSH patch rows replace a row's complete config, so every overlay must restate all keys it relies on.
| Action | DSH pipeline phase | Behavior |
|---|---|---|
deny | tools/pre-execute | Reject before dispatch |
error | tools/execute | Return a structured injected error without running the tool body |
delay | tools/execute | Add abort-aware latency |
abort | tools/execute | Abort a derived signal and restore the upstream signal |
block | tools/post-execute | Run the tool, then withhold its result |
Each decision emits tool-chaos/decision. The optional reporter converts it to a stable stderr JSONL boundary prefixed with DSH_TOOL_CHAOS_EVENT .
The CLI writes a JSON source-of-truth report and a Markdown summary. Reports include configuration hashes, bounded process metadata, parsed chaos decisions, assertions, verdict, and a reproduction command. Raw command output is excluded unless explicitly requested.
| Verdict | Meaning |
|---|---|
PASS | The intended fault occurred and the declared recovery behavior is supported by evidence |
FAIL | The intended fault occurred and at least one recovery assertion failed |
INCONCLUSIVE | Baseline, installation, matching, safety, or observation evidence is incomplete |
Exit code alone is never sufficient evidence of resilience.
dsh-tool-chaos doctor [--profile name] [--json]
dsh-tool-chaos setup [--profile name] [--source npm-or-path] [--codex] [--json]
dsh-tool-chaos init [path] [--preset retry|timeout|nested|result-block|deny] [--force]
dsh-tool-chaos run --patch file --task prompt [--mode baseline|dry-run|experiment]
[--expect regex] [--profile name] [--json]
dsh-tool-chaos install-plugin [--profile name] [--source npm-or-path] [--json]
dsh-tool-chaos install-skill [--global | --project path]
dsh-tool-chaos install-codex-plugin [--dir path] [--json]
dsh-tool-chaos verify [--json]
dsh-tool-chaos prompt [--lang en|zh] <requirement...>
dsh-tool-chaos codex [--cwd path] [--json] <requirement...>
Automation should consume --json instead of scraping human-readable output.
The repository contains a Codex marketplace plugin and an open Agent Skill. Install the Codex plugin from a checkout:
codex plugin marketplace add /absolute/path/to/dsh-tool-chaos
codex plugin add dsh-tool-chaos@dsh-tool-chaos
Start a new thread and invoke:
$dsh-chaos-test Prove the retry path survives one deterministic GitHub tool error.
See Codex integration for the source-checkout workflow. npx dsh-tool-chaos ... commands become available only after the npm prerelease is published.
The dsh-tool-chaos/engine export has no Cordis or DSH runtime import:
import { ChaosEngine, resolveConfig } from 'dsh-tool-chaos/engine'
const engine = new ChaosEngine(resolveConfig({
enabled: true,
dryRun: false,
seed: 'ci-retry-v1',
rules: [{
id: 'first-web-error',
tool: 'web_fetch',
action: 'error',
maxInjections: 1,
}],
}))
const decision = engine.decide('execute', {
callId: 'call-1',
rootCallId: 'call-1',
name: 'web_fetch',
arguments: { url: 'https://example.test' },
})
See the engine example.
dryRun: true until the exact intended match is visible.* requires allowGlobalWildcard: true.abort is cooperative; a same-process tool that ignores AbortSignal cannot be hard-killed.Production/shared profiles, external mutations, credentials, broad targeting, or privilege changes require separate safeguards and explicit authorization.
| Surface | Verified state |
|---|---|
| Tests | Strict TypeScript compile and 32 unit/CLI/real-runtime tests pass |
| Node | 24.11.1 locally; GitHub CI passes on 22.19.0 and 24.x |
| DSH | @deepseek-ai/dsh-tools@0.1.0-rc.6 runtime integration passes |
| Tarball consumer | Install, repeated add, config dump, and read-only headless dry-run preview pass |
| GitHub source | Pinned commit install, repeated install, config dump, uninstall, and reinstall pass with pnpm 11.7.0 |
| npm | Not published; no npm availability claim is made |
Evidence was collected on August 14, 2026 with isolated profiles and synthetic read-only input. See Compatibility for the precise matrix and limitations.
npm ci
npm run sync:skill
npm run check
Compatibility claims must be backed by the exact environment and command that passed.