dsh-freeroute
English | 简体中文
Free-tier model aggregation for the
DeepSeek Harness (dsh):
registers a freeroute model provider backed by a pool of free-quota
upstreams (OpenCode Zen / B.AI / OpenRouter / SenseNova built in, plus remote
catalogs and custom gateways; other vendors join via the remote catalog).
Configure any one API key and you are running; when an upstream rate-limits,
dies, or loses its key, requests fail over to the next one before the first
token reaches your session. A local OpenAI-compatible endpoint is included.
Free here means providers' legitimate free tiers, with keys you register
yourself. This is not a shared-account pool and does not bypass paid APIs.
Key management
- Multi-key rotation — free quotas are per-account, so each upstream
accepts up to 8 keys (separated by newlines/commas in the field, or
KEY_2…KEY_8 env vars). AUTH / rate-limit failures cool down that single
key and rotate to the next one before any upstream is penalized. A working key
stays in use until it fails (same account across turns, so provider-side
prompt caches keep hitting); the key rotated to then sticks the same way.
403s that gate a single model ("only available on…", region / plan limits)
are model-level: no key rotation, no upstream cooldown — that model is just
skipped for 30 minutes.
- Privacy by default — the key field shows a read-only
•••••• mask until
you click 显示 (show), which opens the editor prefilled with the full key
ring (one key per line) ready to edit; saving returns straight to the masked
state. Keys live only in the credentials service, never in plaintext config.
- Failure visibility — when a key is detected as dead mid-rotation, the
panel warns 「第 N 把 Key 失效」 (which key failed and why), so a dying
account is obvious without digging through logs.
How it works
- Adapter registration —
llm.registerAdapter(['freeroute'], adapter)
exposes freeroute/auto (automatic failover) and
freeroute/<upstream>/<model> (pinned model) in the model picker.
- Translation — dsh internal messages ⇄ OpenAI Chat Completions SSE, with
reasoning, tool_calls, and usage passed through. Keys live only in the
credentials service, never in plaintext config.
- Health & failover — failures cool down by class (auth/config 10 min,
rate limit 60 s, others exponential backoff capped at 10 min).
auto
ranks candidates in four layers: the anchor model (the one shared by the
most upstreams; declared quality only breaks ties) on every vendor that
fits the current context, largest window first → other models whose window
is no smaller than the chain head → other models that fit the current
context → everything else (sunk, never dropped). Windows are judged per
vendor, not per model, since the same model ships with very different
windows on different gateways. A pinned single model degrades to the auto chain when all
of its providers fail (or sit in cooldown) instead of aborting the turn —
picking one model is a preference, not exclusivity. An upstream that emits
no token within 90 s (queued, or only sending SSE keep-alives) also fails
over (the last candidate gets 600 s), and slow-to-first-token providers sink
to the end of their layer. Once tokens flow, a stream is only cut after
120 s with no new content. Tune with FREEROUTE_FIRST_TOKEN_SECONDS
(0 disables) and FREEROUTE_CONTENT_IDLE_SECONDS. Keys rotate
individually on AUTH / rate-limit, and a per-key failure is surfaced in the
panel.
- Three-layer config — builtin catalog → remote catalog (JSON hosted on
Cloudflare Pages/R2 for ship-free updates; native format or models.dev
api.json, zero-cost models auto-filtered) → user patch persisted under
the free-proxy settings namespace.
- Local endpoint —
http://127.0.0.1:<port>/freeroute/v1 is an
- Global proxy (off by default): set one
http://127.0.0.1:7890-style
proxy under Settings → Models → Free → Advanced; every upstream without
its own proxy (chat requests and model probing) goes through it. Per-upstream
custom.proxy and catalog-declared proxy win over the global one; catalog
sync always connects directly. One entry covers Clash/v2ray users — no
per-upstream configuration needed.
OpenAI-compatible base URL (chat/completions with streaming, tools and
usage; models; health). No API key required, CORS enabled — any other
agent or client (CLI or browser) can ride the same free pool.
Settings panel
dsh's built-in Settings -> 模型 (models) page is reversibly wrapped: the
title and intro stay in place, and a 默认 | 免费 tab bar sits right below
the intro (structure mirrors the Settings -> 插件 page: underline tabs, ARIA
roles, arrow-key navigation, visited panels stay mounted):
- 默认 keeps the stock model settings untouched;
- 免费 hosts the full freeroute panel in the same shape as the Plugins
settings page — one expandable card per upstream (header: name, status
summary, priority controls, enable toggle; click to expand keys /
connectivity test / model probe / signup links), with Models and Advanced
(global proxy + remote catalog) as cards of the same form. Non-standard
gateways are supported in the config file via
custom.chatPath
(override /chat/completions) and custom.requestExtra (extra scalar body
fields; model: null omits the model field — e.g. GMI's /autoroute).
There is no sibling freeroute item in the settings nav (a standalone section
appears only if the host ships no wrappable models entry). The /freeproxy
command prints a text status.
Per-message model attribution
In auto mode the UI only ever shows "⚡ Auto", so you cannot tell who actually
answered. Every reply now carries a ⚡ <served model> chip to the right of
its timestamp (after 13:17); hovering shows the upstream name and the
requested model (auto). It reuses the action row's existing hover reveal, so
no new interaction is introduced. After a failover you can see at a glance
which upstream and model served that message.
Attribution lives in host memory only — it is not written to the session
log (dsh's persistence read path rejects unregistered event types), so
history stops showing chips after a dsh web restart; new messages are
unaffected.
Let other agents / tools use the pool (always auto)
While dsh web runs, the plugin serves an OpenAI-compatible endpoint on the
same port — any agent or tool that accepts a custom base URL can reuse the free
pool:
- Base URL:
http://127.0.0.1:<dsh web port>/freeroute/v1
(default port 3080; the Free panel footer shows the exact URL)
- API key: none needed (loopback only). Tools that demand a non-empty key
accept a placeholder like
sk-freeroute
- Model: set
auto, or omit it entirely (auto is the default) — picks
enabled free upstreams by priority, fails over across the whole chain on
errors / rate limits / pre-first-token drops, and degrades single-model
requests to chain fallback automatically
- Capabilities:
stream, tools (function calling), stop,
temperature, max_tokens pass through; CORS is open for browser extensions
- Health:
GET /freeroute/health; model list:
GET /freeroute/v1/models (auto is the first entry)
Minimal curl (no model field = auto):
curl http://127.0.0.1:3080/freeroute/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"messages":[{"role":"user","content":"hello"}]}'
Notes: the endpoint lives as long as the dsh web process (same machine);
configure at least one upstream key in the Free panel so the auto chain has a
usable node.
Install
dsh plugin --profile web add dsh-freeroute
Requirements: Node.js ≥ 18; the host provides llm / timer; settings /
credentials / subprocess / webServer / commands / agentDefaultModel
are optional and degrade gracefully when absent.
Development
Layered sources (see src/README.md):
src/ is the single hand-edited authority, split into src/host/ (20
fragments: constants → catalog parse → context/state → keys/probe/models/
health → transport/router/http → takeover/rpc/commands/endpoint/boot) and
src/client/ (10 fragments: styles/context → panel state/header/upstreams/
advanced/models → models page/integration/plugin tail).npm run build:dynamic assembles src/** into freeroute-dynamic/ {host,client}.js (single-function-body form for the dynamic plugin loader,
with the version injected from package.json).freeroute-dynamic/ carries a 137-assertion integration suite:
node freeroute-dynamic/test/integration.mjs.npm run build:static mechanically generates lib/ from those assembled
sources (the dynamic RPC layer is swapped for a Typert Remote + Connection
carrier) — never hand-edit just one side; npm run build:static runs both.npm run lint && npm test — lint plus a smoke test on a real cordis root.- Module-instance rule:
@deepseek-ai/dsh-typert-protocol must resolve to
the SAME physical copy the dsh host uses. The host gateway reads Remote
method markers from a WeakMap private to ITS copy of the module; a private
pnpm copy in the plugin workspace registers into a different WeakMap, so
/api/freeroute/* silently registers zero routes (client sees HTTP 404,
no error anywhere). Since v0.7.3 lib/index.js resolves the host's own copy
at load time (running dsh CLI entry → global npm layout → fallback), so no
postinstall, symlink, or host version is required. If you ever see
transport failure for /api/freeroute/state: HTTP 404 on an older version,
upgrading to ≥0.7.3 fixes it permanently.
Honest limits
This plugin aggregates free tiers you register yourself. If you need
multi-tenant billing or channel management, run uni-api, new-api, or LiteLLM
yourself and mount it as a custom upstream here — the two complement each
other instead of reinventing.
Web panel (optional)
lib/client.js registers a "Free Models" settings section in dsh web
with the same ranking and one-click default. dsh's web frontend ships
pre-built, so surfacing the panel requires adding this package to the
dsh-web-frontend client bundle and rebuilding it.
free-models-preview.html mirrors the panel standalone (serve the repo
root over HTTP and open it) for iteration without that rebuild.
License
MIT © dushaobindoudou
