dsh-model-failover
Two-level model circuit breaker with failover for DeepSeek Harness. When a model (or a whole provider) starts failing repeatedly, the plugin opens a circuit and routes the next model request to a configured fallback — no core changes, installable with dsh plugin.
What it does
- Model circuit — a
provider/model route opens after modelCircuitThreshold failures inside burstWindowMs.
- Platform circuit — a provider opens when
platformCircuitThreshold distinct models under it are open at once (a platform outage usually takes down every model on it).
- Failover — the next request goes to the first healthy fallback in
fallbacks; the switch is recorded by the loop itself (request/header change) and optionally announced as a user-visible message.
- Recovery probes — an open model circuit is probed after
modelCooldownMs with a tiny real call; a successful probe closes the circuit, a failed one extends the cooldown.
- Composes with
llm-retry — per-request backoff retries stay owned by the bundled llm-retry policy; the breaker observes the failures that escape it, so transient blips that recover after a retry never trip a circuit.
Install
dsh plugin --profile web add dsh-model-failover
Then configure fallbacks (the only field you must set) and, if needed, the thresholds in your profile cordis.patch.yml — see the plugin row in cordis.patch.yml for the full default config.
A companion skill, configure-model-failover, walks the agent through setting the fallback models (AI probes the current model config, writes the fallback override, then asks you to confirm). It is installed automatically with the plugin: the package ships skills/configure-model-failover/SKILL.md and the plugin registers it as a bundled skill whenever the skills service is present — no copy step needed. For a standalone install (profile without the plugin), copy it into ~/.dsh/skills/configure-model-failover/ (user-level skills are picked up live).
How it works
The plugin decorates two agent-loop waterfalls (both official extension points, no core changes):
| Waterfall | Role |
|---|
agent/request-error | Records failures whose code is in tripCodes into the circuit breaker, then delegates through next() so llm-retry still owns retries. |
agent/request | await next() for the resolved config, then returns the healthy primary route, or the first healthy fallback when the primary's circuit is open. |
request ──> agent/request ──> primary (mock/m1) ──> fail ×2 ──> circuit open
│
next request ──> agent/request ──> primary open ──> fallback (mock2/m2) ✔
│
probe after cooldown ──> success ──> circuit closed
Configuration
| Field | Default | Meaning |
|---|
enabled | true | Master switch. |
fallbacks | [] | Ordered fallback routes {provider, model}; must point at providers with a registered adapter. |
tripCodes | RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, QUOTA, EMPTY_RESPONSE | Failure codes that count toward a circuit; e.g. AUTH/INVALID_CREDENTIAL stay terminal. |
modelCircuitThreshold | 2 | Failures inside the burst window that open a model circuit. |
modelCooldownMs | 60000 | Cooldown before an open model circuit is probed. |
platformCircuitThreshold | 2 | Distinct open models that open the whole provider. |
platformCooldownMs | 120000 | Provider-wide cooldown. |
burstWindowMs | 300000 | Failures older than this start a fresh burst. |
enableProbe | true | Probe open models after cooldown to recover circuits. |
probeMaxTokens | 8 | Output cap for probe calls. |
stripReasoningEffort | true | Drop the primary's reasoning effort when failing over (fallbacks may not support it). |
notifyUser | true | Append a user-visible message when a route switches. |
Events
Plugin-defined (emit) events, typed via the @deepseek-ai/cordis augmentation in src/types.ts:
model-failover/circuit-opened — {provider, model, level: 'model' | 'platform'}model-failover/circuit-closed — {provider, model, level: 'model'}model-failover/failover — {from, to, agentId}model-failover/probe — {provider, model, ok, message?}
Known Limitations and Deferred Work
- Process-local state — circuit state lives in memory and resets on plugin reload (like every harness registry). Cross-instance sharing is deferred.
- Agent-loop calls only —
agent/request covers the main conversation loop. Auxiliary calls (session-title, compaction, hand-built ctx.llm.stream) are not routed.
retryPolicy.mode: 'always' — the bundled llm-retry never delegates a failure to this breaker in that mode, so failover stays idle by design (the operator chose unbounded retries).- No context-window adaptation — a fallback with a smaller context window may hit
CONTEXT_WINDOW_EXCEEDED; set stripReasoningEffort and pick compatible fallbacks.
- No platform probe — the platform circuit recovers by cooldown expiry; only model circuits are probed.
License
MIT
