dsh-quality-review
English | 中文
AI response quality review for DeepSeek Harness (DSH): audits each finished assistant turn with an independent reviewer model and steers the agent to fix unreasonable output (up to 2 review rounds).
How it works
On agent/turn-stopping — the moment a turn is about to close — the plugin takes the assistant's latest visible reply, audits it with a reviewer model, and when the verdict is fail it steers the agent with a concrete fix request so the turn stays open and the model revises. Each turn allows at most maxRounds steered follow-ups (default 2), which is the hard loop guard.
Reviewed aspects (each can be toggled):
- Factual accuracy — obvious factual errors and fabricated information
- Completeness — any part of the user's question left unanswered
- Logical consistency — contradictory reasoning and unsupportable conclusions
- Instruction following — violations of explicit user format/language/constraint requirements
Install
dsh plugin --profile web add dsh-quality-review
Or add it to your profile's package.json dependencies and list it under dsh.profile.bundles, then restart dsh web.
Configuration
Every field has a safe default, so the bundle row can be inserted with no config at all. Override individual keys from your profile's cordis.patch.yml:
- id: quality-review
config:
enabled: true # master switch; the plugin stays loaded but inert when false
reviewer:
provider: '' # reviewer provider id; empty reuses the agent's own provider
model: '' # reviewer model id; empty reuses the agent's own model
maxRounds: 2 # max steered fix rounds per turn (loop guard, max 5)
aspects:
factualAccuracy: true
completeness: true
logicalConsistency: true
instructionFollowing: true
Development
npm install
npm run build # tsc — compiles src/ to lib/
The published package ships the compiled lib/ output; src/ holds the TypeScript sources.
License
MIT
