dsh-llm-auto-route
dsh-llm-auto-route is a community Cordis plugin for DeepSeek Harness. It discovers which already-configured dsh-llm-pi-ai route should handle a request, explains the decision, and performs failover only before the first visible output.
It is deliberately a routing policy layer. It does not implement HTTP protocols, ship provider SDKs, or register openai, anthropic, deepseek, or other adapter routes. The official @deepseek-ai/dsh-llm-pi-ai plugin remains the owner of those routes.
Community project: this package is not an official DeepSeek Harness package and does not imply DeepSeek endorsement.
Requirements
- Node.js
>=22.19.0
- DeepSeek Harness
0.1.0-rc.5 or a compatible 0.1.x release
@deepseek-ai/dsh-llm-pi-ai configured with the route names you want to select
The package is tested against the npm 0.1.0-rc.6 companion packages while keeping the peer range compatible with rc.5.
Install
pnpm add dsh-llm-auto-route
The official base bundle already contains @deepseek-ai/dsh-llm-pi-ai. If you compose plugins manually, install and load that official adapter before this package. Load the shipped cordis.patch.yml with the normal DeepSeek Harness/Cordis composition command used by your deployment.
The patch adds one plugin named llm-auto-route; it does not add or replace any official adapter route.
Configure the official adapter first
The route names in this plugin must already be registered by dsh-llm-pi-ai. The following is an abbreviated official-adapter configuration:
- id: llm
name: '@deepseek-ai/dsh-llm-pi-ai'
config:
providers:
deepseek:
apiKeyEnv: DEEPSEEK_API_KEY
openai:
apiKeyEnv: OPENAI_API_KEY
anthropic:
apiKeyEnv: ANTHROPIC_API_KEY
ollama:
baseURL: http://127.0.0.1:11434/v1
api: openai-completions
models:
- id: llama3.1
contextWindow: 131072
maxTokens: 8192
vllm:
baseURL: http://127.0.0.1:8000/v1
api: openai-completions
models:
- id: local-model
contextWindow: 32768
maxTokens: 4096
openai-compatible:
apiKeyEnv: GATEWAY_API_KEY
baseURL: https://gateway.example.test/v1
api: openai-completions
models:
- id: gateway-model
contextWindow: 65536
maxTokens: 8192
dsh-llm-pi-ai owns credentials, model metadata, transport, and stream conversion. The auto-route plugin only sees the route directory and selects one of those route keys.
Automatic selection
An automatic request uses provider: auto, or omits the provider when the plugin configuration's provider token is auto:
const options = {
provider: 'auto',
model: 'deepseek-chat',
messages,
}
The fixed default precedence is:
explicit → provider_env → base_url → model_prefix
- A non-
auto provider is always preserved. Explicit requests are never silently rerouted.
DEEPSEEK_API_KEY, OPENAI_API_KEY, and ANTHROPIC_API_KEY are detection signals only; the key value is never written to a log.LLM_BASE_URL selects the generic openai-compatible rule. 11434 identifies Ollama and 8000 identifies a vLLM/OpenAI-compatible local endpoint.deepseek-*, gpt-*, o1-*, o3-*, and claude-* provide model-prefix hints.- A candidate is usable only when the official adapter has registered the same route.
- Equal-priority candidates at the same stage return
AMBIGUOUS_ROUTE; the plugin does not guess.
- A route's
defaultModel is used only when a matching rule supplies one and the request omits its model.
Every selection can be explained without exposing credentials:
已选择 deepseek/deepseek-chat；原因：发现 DEEPSEEK_API_KEY
Use an explicit provider or a route priority when an environment intentionally contains multiple credentials.
Configuration
The shipped patch contains the default rules. An application can override them through its Cordis configuration:
provider: auto
precedence:
- explicit
- provider_env
- base_url
- model_prefix
healthCheck:
mode: adaptive # off | adaptive | probe
timeoutMs: 3000
cacheTtlMs: 30000
failover:
enabled: true
maxAttempts: 3
diagnostics: info # silent | error | info
routes:
deepseek:
apiKeyEnv: DEEPSEEK_API_KEY
defaultModel: deepseek-chat
priority: 10
openai-compatible:
apiKeyEnv: GATEWAY_API_KEY
baseURLEnv: LLM_BASE_URL
modelPrefixes: [gateway-]
Route fields are hints, not adapter configuration:
| Field | Meaning |
|---|
provider | Registered route id to return; defaults to the routes key. |
apiKeyEnv | Non-empty environment variable used as a provider-env signal and for discovery. |
baseURL / baseURLEnv | Exact or user-supplied endpoint hint. baseURLEnv also supports unknown OpenAI-compatible hosts. |
baseURLPatterns | Additional normalized URL prefixes. |
ports | Local ports that identify this route when a base URL is supplied. |
modelPrefixes | Model id prefixes for automatic selection. |
defaultModel | Model to use when the request does not name one. |
priority | Tie breaker within one matching stage; higher wins. |
Do not put an API key directly in route configuration. Use the official adapter's credential reference, normally apiKeyEnv, and keep this plugin's apiKeyEnv aligned with it.
Health checks and failover
adaptive health checks reuse the official dsh-llm model-discovery seam when it is available, with a bounded timeout and an in-memory cache. If a deployment cannot expose discovery, route/model resolution is used as the local fallback and the real stream remains the final availability check. off skips preflight checks. probe asks the discovery seam whenever the adapter exposes it.
Failover is intentionally conservative:
- it is allowed only before text, reasoning, tool-call, or block output has been emitted;
- buffered protocol metadata from a failed attempt is discarded before trying the next route;
- aborts, explicit providers, configuration errors, and requests that already emitted output are not retried;
- a later provider never receives a partial assistant response from an earlier provider.
The plugin observes agent/request, agent/request-error, and llm/stream. It returns a new immutable request configuration and never mutates a frozen request object.
Public API
The package exports AutoRouteConfig, RouteRule, RouteDecision, MatchStage, and the pure helpers normalizeConfig, normalizeBaseURL, and resolveRoute:
import { normalizeConfig, resolveRoute } from 'dsh-llm-auto-route'
const decision = resolveRoute(normalizeConfig(), {
model: 'deepseek-chat',
env: { DEEPSEEK_API_KEY: 'present' },
registeredProviders: new Set(['deepseek']),
})
if (decision.kind === 'matched') {
console.log(decision.candidate.provider, decision.candidate.model, decision.stage)
}
AutoRouteError.code is stable for AMBIGUOUS_ROUTE, MISSING_MODEL, NO_CANDIDATE, and NO_REGISTERED_ROUTE.
Troubleshooting
NO_REGISTERED_ROUTE — the route exists in this plugin's default hints but not in ctx.llm.listProviders(). Add the same key under the official adapter's providers configuration.
AMBIGUOUS_ROUTE — more than one candidate matched at the same stage and priority. Set provider explicitly, remove an unused environment variable, or give one route a higher priority.
MISSING_CREDENTIAL / INVALID_CREDENTIAL — detection and the official adapter must agree on the environment variable. Check the variable name, not its value in logs or issue reports.
Local gateway is not selected — set LLM_BASE_URL, or provide a URL containing port 11434 (Ollama) or 8000 (vLLM). The official adapter still needs a route with that key and a model catalog.
A response was not retried — this is expected after the first text, reasoning, tool-call, or block output, after cancellation, for an explicit provider, or for a configuration failure. Retrying then could duplicate or splice an assistant response.
Compatibility and project status
This is an independent ecosystem plugin for the DeepSeek Harness developer preview. It follows the current upstream guidance for community plugins, dsh-plugin topics, Discussions announcements, and standalone repositories. It does not open a pull request against the official Harness repository.
Upstream references:
License
MIT. See LICENSE.
