dsh-llm-gate
用于 DeepSeek Harness LLM 请求的按提供商并发限制
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add dsh-llm-gate@0.1.0Llm Gate 以 dsh-llm-gate 发布，当前版本为 0.1.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
Per-provider concurrency gate for DeepSeek Harness model requests.
If a provider can only serve a fixed number of requests at once (e.g a local llama-server with --parallel 1), every extra request is deferred by the server with nothing sent back. The client cannot tell "waiting for a slot" from "dead", and Node HTTP layer times out after 300 seconds with terminated. In practice this happens when there is overlap between a subagent and the main agent or compaction and the agent.
This plugin holds surplus requests inside dsh instead. A request waits in a FIFO queue before any HTTP request is made so no timeout is running while it waits. When a slot frees, the next request is dispatched.
dsh plugin --profile web add dsh-llm-gate
Then configure the providers to gate in ~/.dsh/profiles/web/cordis.patch.yml:
- id: llm-gate
config:
providers:
llamacpp:
maxConcurrent: 1
maxQueued: 16
queueTimeoutMs: 3600000
The provider key is the route name from your llm-pi-ai.providers (or other adapter) settings. Providers not listed are not gated. Restart dsh web and open a new session.
Check the composed config with dsh --profile web --dump-config.
| Setting | Required | Meaning |
|---|---|---|
maxConcurrent | yes | Requests allowed in flight to this provider. For llama.cpp, match --parallel. |
maxQueued | no | Requests allowed to wait. Beyond this, a request fails at once with QUEUE_FULL. Default: unlimited. |
queueTimeoutMs | no | Longest a request may wait for a slot before failing with QUEUE_TIMEOUT. Default: wait indefinitely. |
Queue failures end the turn with the code shown. They are not retried by dsh-llm-retry.
The plugin prints a line to the dsh terminal only when a request has to wait:
llm-gate: llamacpp session=a61e6e40 queued (depth 1)
llm-gate: llamacpp session=a61e6e40 dispatched after 5730ms
purpose=compaction or purpose=session-title is added for auxiliary requests. Requests that get a slot immediately print nothing.
--parallel 2 --kv-unified) and raise maxConcurrent to match.streamIdleTimeoutMs because the adapter is not called until the slot is acquired. You still need streamIdleTimeoutMs large enough for your prompt processing time (see the llm-pi-ai provider settings).llm service; hooks the llm/stream waterfall, so it covers every model request in the host: agents, subagents, compaction, and title generation.MIT