dsh-llm-auto-route
DeepSeek Harness 的服务提供商发现、匹配、健康检查和输出前故障转移。
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add dsh-llm-auto-route@0.1.0Llm Auto Route 以 dsh-llm-auto-route 发布，当前版本为 0.1.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
dsh-llm-auto-route is a community Cordis plugin for DeepSeek Harness. It discovers which already-configured dsh-llm-pi-ai route should handle a request, explains the decision, and performs failover only before the first visible output.
It is deliberately a routing policy layer. It does not implement HTTP protocols, ship provider SDKs, or register openai, anthropic, deepseek, or other adapter routes. The official @deepseek-ai/dsh-llm-pi-ai plugin remains the owner of those routes.
Community project: this package is not an official DeepSeek Harness package and does not imply DeepSeek endorsement.
>=22.19.00.1.0-rc.5 or a compatible 0.1.x release@deepseek-ai/dsh-llm-pi-ai configured with the route names you want to selectThe package is tested against the npm 0.1.0-rc.6 companion packages while keeping the peer range compatible with rc.5.
pnpm add dsh-llm-auto-route
The official base bundle already contains @deepseek-ai/dsh-llm-pi-ai. If you compose plugins manually, install and load that official adapter before this package. Load the shipped cordis.patch.yml with the normal DeepSeek Harness/Cordis composition command used by your deployment.
The patch adds one plugin named llm-auto-route; it does not add or replace any official adapter route.
The route names in this plugin must already be registered by dsh-llm-pi-ai. The following is an abbreviated official-adapter configuration:
- id: llm
name: '@deepseek-ai/dsh-llm-pi-ai'
config:
providers:
deepseek:
apiKeyEnv: DEEPSEEK_API_KEY
openai:
apiKeyEnv: OPENAI_API_KEY
anthropic:
apiKeyEnv: ANTHROPIC_API_KEY
ollama:
baseURL: http://127.0.0.1:11434/v1
api: openai-completions
models:
- id: llama3.1
contextWindow: 131072
maxTokens: 8192
vllm:
baseURL: http://127.0.0.1:8000/v1
api: openai-completions
models:
- id: local-model
contextWindow: 32768
maxTokens: 4096
openai-compatible:
apiKeyEnv: GATEWAY_API_KEY
baseURL: https://gateway.example.test/v1
api: openai-completions
models:
- id: gateway-model
contextWindow: 65536
maxTokens: 8192
dsh-llm-pi-ai owns credentials, model metadata, transport, and stream conversion. The auto-route plugin only sees the route directory and selects one of those route keys.
An automatic request uses provider: auto, or omits the provider when the plugin configuration's provider token is auto:
const options = {
provider: 'auto',
model: 'deepseek-chat',
messages,
}
The fixed default precedence is:
explicit → provider_env → base_url → model_prefix
auto provider is always preserved. Explicit requests are never silently rerouted.DEEPSEEK_API_KEY, OPENAI_API_KEY, and ANTHROPIC_API_KEY are detection signals only; the key value is never written to a log.LLM_BASE_URL selects the generic openai-compatible rule. 11434 identifies Ollama and 8000 identifies a vLLM/OpenAI-compatible local endpoint.deepseek-*, gpt-*, o1-*, o3-*, and claude-* provide model-prefix hints.AMBIGUOUS_ROUTE; the plugin does not guess.defaultModel is used only when a matching rule supplies one and the request omits its model.Every selection can be explained without exposing credentials:
已选择 deepseek/deepseek-chat；原因：发现 DEEPSEEK_API_KEY
Use an explicit provider or a route priority when an environment intentionally contains multiple credentials.
The shipped patch contains the default rules. An application can override them through its Cordis configuration:
provider: auto
precedence:
- explicit
- provider_env
- base_url
- model_prefix
healthCheck:
mode: adaptive # off | adaptive | probe
timeoutMs: 3000
cacheTtlMs: 30000
failover:
enabled: true
maxAttempts: 3
diagnostics: info # silent | error | info
routes:
deepseek:
apiKeyEnv: DEEPSEEK_API_KEY
defaultModel: deepseek-chat
priority: 10
openai-compatible:
apiKeyEnv: GATEWAY_API_KEY
baseURLEnv: LLM_BASE_URL
modelPrefixes: [gateway-]
Route fields are hints, not adapter configuration:
| Field | Meaning |
|---|---|
provider | Registered route id to return; defaults to the routes key. |
apiKeyEnv | Non-empty environment variable used as a provider-env signal and for discovery. |
baseURL / baseURLEnv | Exact or user-supplied endpoint hint. baseURLEnv also supports unknown OpenAI-compatible hosts. |
baseURLPatterns | Additional normalized URL prefixes. |
ports | Local ports that identify this route when a base URL is supplied. |
modelPrefixes | Model id prefixes for automatic selection. |
defaultModel | Model to use when the request does not name one. |
priority | Tie breaker within one matching stage; higher wins. |
Do not put an API key directly in route configuration. Use the official adapter's credential reference, normally apiKeyEnv, and keep this plugin's apiKeyEnv aligned with it.
adaptive health checks reuse the official dsh-llm model-discovery seam when it is available, with a bounded timeout and an in-memory cache. If a deployment cannot expose discovery, route/model resolution is used as the local fallback and the real stream remains the final availability check. off skips preflight checks. probe asks the discovery seam whenever the adapter exposes it.
Failover is intentionally conservative:
The plugin observes agent/request, agent/request-error, and llm/stream. It returns a new immutable request configuration and never mutates a frozen request object.
The package exports AutoRouteConfig, RouteRule, RouteDecision, MatchStage, and the pure helpers normalizeConfig, normalizeBaseURL, and resolveRoute:
import { normalizeConfig, resolveRoute } from 'dsh-llm-auto-route'
const decision = resolveRoute(normalizeConfig(), {
model: 'deepseek-chat',
env: { DEEPSEEK_API_KEY: 'present' },
registeredProviders: new Set(['deepseek']),
})
if (decision.kind === 'matched') {
console.log(decision.candidate.provider, decision.candidate.model, decision.stage)
}
AutoRouteError.code is stable for AMBIGUOUS_ROUTE, MISSING_MODEL, NO_CANDIDATE, and NO_REGISTERED_ROUTE.
NO_REGISTERED_ROUTE — the route exists in this plugin's default hints but not in ctx.llm.listProviders(). Add the same key under the official adapter's providers configuration.
AMBIGUOUS_ROUTE — more than one candidate matched at the same stage and priority. Set provider explicitly, remove an unused environment variable, or give one route a higher priority.
MISSING_CREDENTIAL / INVALID_CREDENTIAL — detection and the official adapter must agree on the environment variable. Check the variable name, not its value in logs or issue reports.
Local gateway is not selected — set LLM_BASE_URL, or provide a URL containing port 11434 (Ollama) or 8000 (vLLM). The official adapter still needs a route with that key and a model catalog.
A response was not retried — this is expected after the first text, reasoning, tool-call, or block output, after cancellation, for an explicit provider, or for a configuration failure. Retrying then could duplicate or splice an assistant response.
This is an independent ecosystem plugin for the DeepSeek Harness developer preview. It follows the current upstream guidance for community plugins, dsh-plugin topics, Discussions announcements, and standalone repositories. It does not open a pull request against the official Harness repository.
Upstream references:
MIT. See LICENSE.