dsh-auto-review-model-router
Route DSH's experimental auto-review safety gate to a model you choose, so a
premium agent model does not pay for its own safety reviews.
# <profile>/cordis.patch.yml
- insert:
- id: auto-review-model-router
name: dsh-auto-review-model-router
config:
reviewModel: deepseek-official/deepseek-flash
With that entry every review runs on deepseek-official/deepseek-flash while the
agent keeps claude-opus-4, gpt-6-astra, or whatever else the session selected.
The problem it solves
@deepseek-ai/dsh-experimental-auto-review reviews each gated tool call by
calling ctx.llm.stream() on the session's own route: it copies provider
and model out of the session request header
(snapshotAutoReview() in packages/experimental/auto-review/src/index.ts).
There is no reviewModel field in the shipped plugin, so an expensive agent
model is charged for one extra full review request per gated call.
How it works
The plugin wraps two service methods at activation: llm.stream always, and
llm.prepareCall unless wrapPrepareCall is off.
- A request is recognized as an auto-review call when it is hand-built
(it does not carry the agent loop's own-request mark) and its
system
prompt starts with auto-review's fixed REVIEW_POLICY opening. Only such a
request is ever rewritten.
- It is copied onto
reviewModel, and the source route's reasoningEffort and
maxTokens are dropped. Both are properties of the agent's model: a
reasoning effort the target does not declare makes the harness throw
UNSUPPORTED_REASONING_EFFORT, and an inherited output cap can exceed the
target's limit. Either would fail every review closed.
- Everything else — the review prompt, its policy system prompt,
temperature,
and the {"risk":…,"decision":…} JSON protocol the gate parses — is
unchanged. Only the model that produces the verdict changes.
- A review that fails transiently is retried on the same route
(
retryAttempts, default 4). The gate calls llm.stream directly, outside
the agent loop, so the harness retry plugin (an agent/request-error
listener) never sees a review: without this, one dropped connection denies
the tool call.
Why retries need their own classification
Only the codes the harness itself treats as transient are retried:
EMPTY_RESPONSE, RATE_LIMIT, SERVER, TRANSPORT. TIMEOUT is deliberately
excluded — the DeepSeek adapter raises it from an idle watchdog, so retrying
it multiplies a silent wait by the attempt count while the auto-review deadline
drains. ABORTED (the deadline, or the caller cancelling) and configuration
errors such as AUTH are not fixed by trying again.
A failed attempt's terminal finish chunk is held back: the gate's reader ends
the review on any terminal finish, so yielding it would end the review on the
first transport blip. Reasoning chunks stay live (the reader ignores reasoning
content), so a retried review exposes more than one attempt's reasoning while the
verdict is still parsed from the final text block alone.
Why the wrappers, and not an llm/stream listener
Two seams were tried first and both are dead ends on a real runtime:
- Mutating the request inside an
llm/stream listener throws: auto-review
freezes it (deepFreeze) before streaming.
- Returning
next(replacement) from a listener never reaches the adapter:
Cordis's EventsService.waterfall invokes the innermost callback with the
argument list it closed over.
- Writing
ctx.llm.stream = wrapper is also ineffective: ctx.llm is a
per-read traceable proxy, and its property overlay shadows what you write.
The wrappers are therefore installed on the object that owns each method in the
service's prototype chain (LlmRuntime.prototype in practice), which is the one
seam every consumer actually calls. Both are removed when the plugin's fiber
disposes.
Why prepareCall needs its own wrapper
A prepared handle resolves its route during prepareCall and then rejects a
dispatch whose config differs from the resolved one
(INVALID_PREPARED_CALL). The retarget must therefore happen before the call
is prepared; swapping the route in the returned handle cannot work. With
wrapPrepareCall: true (the default) a review prepared that way runs on the
review route, and turning it off restores the session route for that path.
Install
Published on npm as dsh-auto-review-model-router:
# in the profile directory
pnpm add dsh-auto-review-model-router
From a local checkout instead, link it:
// <profile>/package.json
"dependencies": { "dsh-auto-review-model-router": "link:D:/path/to/dsh-auto-review-model-router" }
Then insert it in the profile patch layer, after the auto-review bundle:
# <profile>/cordis.patch.yml
- insert:
- id: auto-review-model-router
name: dsh-auto-review-model-router
config:
reviewModel: deepseek-official/deepseek-flash
Restart the app: patch-layer changes apply at profile startup.
Configuration
| Field | Default | Meaning |
|---|
reviewModel | deepseek-official/deepseek-flash | <provider>/<model> every review runs on. An empty string disables routing. |
systemPromptPatterns | the shipped REVIEW_POLICY opening | Substrings, or /regex/u literals, matched against a request's system prompt. Widen this only if a harness release rewrites that prompt. |
retryAttempts | 4 | Retries after the first attempt of one review call, so up to 5 attempts. 0 disables retrying. Maximum 10. |
retryDelayMs | 500 | Pause before each retry. |
wrapPrepareCall | true | Also route review requests issued through llm.prepareCall. |
dropReasoningEffort | true | Drop the source route's reasoning effort on the review request. |
dropMaxTokens | true | Drop the source route's output cap on the review request. |
logRetargets | true | Log each retargeted review. Retries log a warning regardless. |
Verify
The test suite has two files. test/router.test.js covers the decision surface
with no harness at all; test/live.test.js runs the wrappers over the real
LlmRuntime and skips those checks when neither this package nor
DSH_PROFILE_DIR resolves @deepseek-ai/dsh-llm.
node test/router.test.js
DSH_PROFILE_DIR=<profile dir> node test/live.test.js
Two commands rather than node --test: each file runs its node:test suite
in-process, while node --test spawns a child process per file.
The plugin patches LlmRuntime.prototype, so each live test disposes its own
mount. Skipping that teardown leaks a wrapper into the next test's dispatches.
At runtime each retarget logs one line, and each retry logs a warning:
auto-review-model-router: review request re-routed anthropic/claude-opus-4 -> deepseek-official/deepseek-flash (stream)
auto-review-model-router: review attempt 1/5 failed with TRANSPORT; retrying in 500 ms
Scope and limits
- Only requests recognized as auto-review calls change route. A false negative
costs nothing (the review runs exactly as before); a false positive is bounded
to one extra object and one changed route for one hand-built request.
- The standalone
dsh-auto-review plugin needs none of this: its reviewerModel
config field already sets the reviewer's model directly.
- DSH public APIs are pre-stable. A release that changes the
REVIEW_POLICY
opening, LlmRuntime.prototype's method names, or the prepared-handle contract
needs a matching update here. Detection is configurable
(systemPromptPatterns) precisely for the first case, and the plugin logs a
warning instead of failing the profile boot when a method it needs is missing.
License
MIT
