dsh-llm-rate-limit
English | 中文
A DeepSeek Harness (DSH) plugin that prevents avoidable API rate-limit errors by pacing LLM requests before they reach the provider. It provides per-provider RPM limits, optional token budgets, concurrency control, bounded FIFO queuing, and adaptive cooldown for DeepSeek API, Volcengine Ark, and other DSH providers.
Use it when parallel agents, subagents, retries, or background requests are producing HTTP 429 errors, provider throttling, or traffic bursts.
Install from npm
Install the latest release into the Web profile:
dsh plugin --profile web add dsh-llm-rate-limit
dsh web
Pin a version for reproducible environments:
dsh plugin --profile web add dsh-llm-rate-limit@0.1.1
Install separately for Headless:
dsh plugin --profile headless add dsh-llm-rate-limit
GitHub installation is also supported:
dsh plugin --profile web add github:Asong6824/dsh-llm-rate-limit#v0.1.1
The bundled default protects deepseek-official with 30 requests per minute, burst 1, two concurrent requests, and a bounded queue.
Features
- Provider-scoped requests-per-minute token buckets with configurable burst capacity.
- Optional estimated-token-per-minute budgets with actual-usage reconciliation.
- Concurrency limits and bounded FIFO queues with timeout and cancellation.
- Adaptive cooldown for provider error codes, HTTP statuses, and
Retry-After.
- Explicit auxiliary-request shedding so background traffic does not block primary work.
- Durable admission wait/start events for DSH session diagnostics.
- Clean lifecycle disposal without abandoning queued or active requests.
- Retry-aware admission: every
dsh-llm-retry attempt is admitted independently; this plugin never retries requests itself.
Configure DeepSeek and Ark
Override the complete llm-rate-limit config in $DSH_HOME/profiles/<profile>/cordis.patch.yml:
- id: llm-rate-limit
config:
providers:
deepseek-official:
requests: { perMinute: 30, burst: 1 }
maxConcurrentRequests: 2
queue: { maxSize: 100, maxWaitMs: 300000, auxiliary: reject }
cooldown:
codes: [RATE_LIMIT, SERVER]
statuses: [429, 529]
initialDelayMs: 500
maxDelayMs: 60000
maxProviderDelayMs: 3600000
jitterRatio: 0.1
volcengine-ark-coding:
requests: { perMinute: 30, burst: 1 }
maxConcurrentRequests: 2
queue: { maxSize: 100, maxWaitMs: 300000, auxiliary: reject }
Provider keys must exactly match GenerateOptions.provider. Optional token limiting adds:
tokens:
perMinute: 1000000
burst: 200000
estimatedOutputTokens: 8192
imageTokens: 1024
tokens.burst must be large enough for one complete request estimate. Omit tokens when a provider should have RPM and concurrency control without a local token ceiling.
How it works
Before each provider call, the plugin reserves request capacity, estimated token capacity, and a concurrency slot. Requests without capacity wait in FIFO order. Provider throttling responses activate a shared cooldown; successful responses reconcile estimated tokens with actual usage. The state is process-local and resets when DSH restarts.
The plugin deliberately does not provide distributed quotas, automatic retries, or provider failover.
Compatibility and links
Development
pnpm install
pnpm run check
MIT
