dsh-protect-tests
DeepSeek Harness 插件，用于检测代理是否通过削弱测试来使其通过：添加跳过、删除断言或测试用例、使用无实际作用的断言、吞掉失败，以及删除测试文件。
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add github:acus9345/dsh-protect-tests#62e886b2fc60d94efecaa80869c77a916a4f6fb6Protect Tests 以 dsh-protect-tests 发布，当前版本为 0.2.3。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
点击下方 Badge 复制 Markdown，粘贴到 README 即可。
验证 package.json 声明的 GitHub 仓库，即可管理这个公开页面。认领后，Hub 会优先安排当前版本的安全扫描，并在通过后公开展示结果。
认领这个 Plugin →English | 中文
Tell an agent "make the tests pass" and sometimes it does exactly that, just not the way you meant. A failing test gets .skip. An assertion gets commented out. expect(total).toBe(42) turns into expect(total).toBeDefined(). The run goes green and the bug is still there.
This DeepSeek Harness plugin looks at every edit to a test file before it's applied, comparing the old text with the new. If the change makes the tests check less, the edit is refused and the model is told what it did: (haha claude is so useful i mean like idk)
dsh-protect-tests blocked this change to src/sum.test.ts because it makes the tests check less:
- replaces specific assertions with existence/truthiness checks (0 → 1): `expect(sum(1, 2)).toBeDefined()`
Tests should keep failing until the code is fixed. Fix the implementation instead, or make
the test more accurate without weakening it. If the test itself is wrong (the requirement
changed, or it tested a bug), tell the user which test you are changing and why, then
repeat the identical call once to confirm.
dsh plugin --profile web add dsh-protect-tests
That's the web profile, the browser UI you start with dsh web. If you run DSH another way, use headless (scripts and CI) or acp (editor integrations) instead. Each profile keeps its own plugin list, so add it to every profile you use. It's on as soon as it's installed. Remove it with dsh plugin --profile web remove dsh-protect-tests.
To try local changes, build a tarball and install that:
git clone https://github.com/acus9345/dsh-protect-tests
cd dsh-protect-tests
npm install
npm pack
dsh plugin --profile web add ./dsh-protect-tests-0.2.0.tgz
it.skip, test.only, xit, xdescribe, @pytest.mark.skip / xfail, @unittest.skip, t.Skip(), @Disabled, #[ignore], and the equivalents in xUnit, RSpec, PHPUnit and XCTest.expect(true).toBe(true), assert True, assertEquals(x, x), assert!(true).catch {} or except AssertionError: pass.toBeDefined(), toBeTruthy(), assert.ok(x), is not None and the like.rm, git rm or unlink on a test file or test folder from the shell."test" in package.json to echo, exit 0, ... || true, or adding --passWithNoTests.On Windows, PowerShell deletes (Remove-Item, del, rd) count too.
Test files are found by name: *.test.*, *.spec.*, *_test.*, test_*.py, *Test.java, conftest.py, anything under tests/, __tests__/, spec/ or e2e/, and so on. You can change the list.
Normal test work isn't affected. Adding tests, adding assertions, renaming, refactoring setup code and fixing a typo in a test name all go through without a word. Refactors that look like losses on paper also go through, for example folding five near-identical tests into one test.each / @pytest.mark.parametrize / t.Run table, or merging three toBe checks into one toEqual({...}) or a helper like expectUser(...).
Sometimes it is. The requirement changed, or the test was asserting the bug. In that case the model is asked to tell you which test it's changing and why, then repeat the same call once to confirm. The repeat goes through. So the guard never fully blocks legitimate work. It just won't let a weakened test slip past without being mentioned in the conversation. If you'd rather approve these yourself, use mode: ask.
It works on patterns, not on meaning. If the model changes toBe(4) to toBe(5) to match a wrong result, that looks like an ordinary edit and gets through. Rewriting the logic of a test so it no longer exercises the bug also gets through. It covers the lazy shortcuts, which turn out to be most of them, but it's no substitute for reading the diff.
The easy way: open Plugins in DSH's sidebar, click dsh-protect-tests, change what you want and press Save. It applies to the next tool call, no restart needed.
Or add a row like this to ~/.dsh/profiles/<profile>/cordis.patch.yml, or to ~/.dsh/cordis.patch.yml for every profile. Keys you leave out keep their defaults.
- id: protect-tests
config:
mode: deny # or `ask` for an approval prompt
testFiles: ['*.test.*', '**/tests/**', '!tests/fixtures/**'] # replaces the default list
minAssertionsRemoved: 2
minTestsRemoved: 1
disabledSignals: [] # skip, fewer-assertions, fewer-tests, trivial-assert,
# commented-assert, swallowed-failure, weaker-assert
guardDeletes: true # watch rm / git rm of test files
guardTestScript: true # watch package.json "test"
allowIdenticalRetry: true # false: a refusal is final
shellTools: [bash, pwsh] # pwsh is the shell DSH uses on Windows
npm install
npm test # detector unit tests
npm run test:e2e # real dsh runs against a scripted local model, no API key needed
The install test runs dsh plugin add, which needs pnpm on your PATH.
MIT licensed. Written by acus9345. Bug reports and ideas are welcome in the issues.