@zhourenke/dsh-agent-rate-limit
English | 中文
Agent loop rate limiter — prevents TPM (Tokens Per Minute) and RPM (Requests Per Minute) limit violations by intercepting the LLM streaming pipeline and adding adaptive delays between requests.
When model providers enforce rate limits (e.g., Alibaba Cloud Bailian's 15,000 RPM + 1,200,000 TPM for deepseek-v4-flash), the agent loop can trigger these limits every few steps, causing errors and interruptions. This plugin solves that by:
- Tracking token usage in a sliding 60-second window
- Estimating input tokens from messages before each request
- Counting output tokens from stream chunks as they arrive
- Adding adaptive delays when approaching TPM or RPM limits
- Exponential backoff on rate-limit errors (auto-retry with
{ kind: 'retry' })
How it works
User input → [agent/request] → [llm/stream*] → LLM API → [agent/request] → ...
↑
Rate limiter intercepts here
┌─ Sliding window (60s FIFO) ──────────────────────┐
│ t0: +5000 tokens (input) │
│ t5: +12000 tokens (input) │
│ t12: +8000 tokens (input) │
│ ... │
│ Current window: 980,000 / 1,200,000 TPM │
│ Remaining: 220,000 tokens → pass through │
│ If approaching limit → delay before next request │
└───────────────────────────────────────────────────┘
The plugin intercepts two Waterfall events:
| Event | Purpose |
|---|
llm/stream | Check rate limits → delay if needed → stream tokens → count output tokens → update window |
agent/request-error | Detect rate-limit errors (429) → return { kind: 'retry' } with exponential backoff |
Installation
This plugin is a DSH profile bundle. The only supported installation method is to place the package folder directly into a DSH profile's node_modules and list it in that profile's dsh.profile.bundles array. No npm link, no pnpm, and no registry access is required.
Find your DSH profile
First, determine which profile you are using:
# List available profiles
Get-ChildItem "$env:USERPROFILE\.dsh\profiles" -Name
Common profiles: web, tui, headless. The profile directory is $env:USERPROFILE\.dsh\profiles\<name>\.
Step 1 — Copy the package folder into the profile
# Create the scoped directory if it does not exist
$target = "$env:USERPROFILE\.dsh\profiles\<name>\node_modules\@zhourenke"
New-Item -ItemType Directory -Force $target
# Copy the whole plugin folder (package.json, cordis.patch.yml, lib/, ...)
Copy-Item -Recurse C:\path\to\dsh-agent-rate-limit "$target\"
The copied tree must contain package.json (with dsh.bundle.patch), cordis.patch.yml, and lib/.
Step 2 — Register the bundle
Edit $env:USERPROFILE\.dsh\profiles\<name>\package.json:
"dsh": {
"profile": {
"bundles": [
"@deepseek-ai/dsh-base",
"@deepseek-ai/dsh-web-app",
+ "@zhourenke/dsh-agent-rate-limit"
]
}
}
No dependencies entry is needed — DSH resolves bundles purely by package name from the profile's node_modules at startup (a dependencies entry only matters to pnpm install, which is not used for this plugin).
Step 3 — Restart DSH
The plugin is loaded on the next DSH startup. When you update the plugin source, re-copy the folder (or use a junction if you prefer live updates) and restart DSH.
Verify the installation
After restarting DSH, check the startup logs for [agent-rate-limit] entries confirming that the rate limiter is active:
dsh web 2>&1 | Select-String "agent-rate-limit"
Expected output:
[agent-rate-limit] Plugin loaded. TPM: 1200000, RPM: 15000, factor: 0.8, window: 60000ms, retryOn429: true
Configuration
| Key | Default | Description |
|---|
windowMs | 60000 | Sliding window size in milliseconds (60s). |
tpmLimit | 1200000 | TPM (Tokens Per Minute) limit. Default matches Alibaba Cloud Bailian deepseek-v4-flash. |
rpmLimit | 15000 | RPM (Requests Per Minute) limit. |
safetyFactor | 0.8 | Safety factor (0.8 = use 80% of the limit, leaving 20% buffer). |
maxBackoffMs | 30000 | Maximum backoff delay in milliseconds (30s). |
retryOn429 | true | When true (default), HTTP 429 responses are silently retried with adaptive backoff — the conversation continues smoothly. Set to false to surface 429 errors to the user. |
maxRetries | 5 | Maximum consecutive 429 retries per burst before giving up and surfacing the error to the user. Prevents an infinite retry loop when the error is permanent (e.g. account quota genuinely exhausted). |
Example: Adjusting for different providers
# In your profile's cordis config or agent preset:
- id: agent-rate-limit
name: '@zhourenke/dsh-agent-rate-limit'
config:
tpmLimit: 2000000 # 2M TPM for a different provider
rpmLimit: 5000 # 5K RPM
safetyFactor: 0.75 # 75% utilization, 25% buffer
retryOn429: true # silently retry on 429 (recommended)
maxRetries: 5 # give up after 5 consecutive 429s
How the rate limiting works
Token estimation
The plugin uses a heuristic to estimate tokens from text:
- CJK characters (Chinese, Japanese, Korean): ~1.5 chars per token
- Other characters (Latin, numbers, etc.): ~3.5 chars per token
This is intentionally conservative — it's better to delay slightly more than to hit the rate limit.
Error recovery
When a rate-limit error is detected (HTTP 429, the typical response from providers like Alibaba Cloud Bailian when TPM or RPM limits are hit), the plugin:
- Records the error and increments the retry count for the current burst
- Returns
{ kind: 'retry' } to tell the agent loop to retry transparently — the user never sees the error
- Applies escalating backoff: 2s → 4s → 8s → 16s → 30s (capped at
maxBackoffMs), reducing the pressure on the API with each retry
- Gives up after
maxRetries consecutive failures (default 5): the error is then surfaced to the user. This prevents an infinite retry loop when the 429 is permanent — e.g. the account's allocated quota is genuinely exhausted ("Allocated quota exceeded, please increase your quota limit")
- When a request finally succeeds, the retry counter resets
Set retryOn429: false in the config if you prefer 429 errors to surface to the user instead of being silently retried.
Sliding window algorithm
The sliding window maintains a FIFO queue of { timestamp, tokens } entries. Before each request:
- Prune entries older than
windowMs (60s)
- Sum remaining tokens = current TPM
- Count entries = current RPM
- If RPM ≥ limit → delay until oldest entry expires
- If TPM ≥ limit → delay until oldest entry expires
- If TPM + estimated input tokens ≥ limit → delay until enough tokens expire
- Apply backoff delay if there were consecutive errors
Rate limit detection
The plugin detects HTTP 429 responses by checking the following in the error:
| Signal | Example |
|---|
| HTTP status code | statusCode: 429 |
| Error code | 429, RATE_LIMITED, QUOTA |
| "rate limit" text | rate limit exceeded, rate_limit |
| "too many requests" | too many requests, please try again later |
| TPM/RPM tokens | TPM limit reached, token limit exceeded |
| "throttle" | request throttled, throttling |
| "quota" | Allocated quota exceeded (Bailian) |
| "429" in message | 429: {...} |
When any of these match, the plugin retries transparently with adaptive backoff by default (retryOn429: true).
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ dsh-agent-rate-limit │
│ │
│ ┌───────────────────────────────────────────────────────────┐ │
│ │ SlidingWindow (module-level state) │ │
│ │ ┌─────────────────────┐ ┌───────────────────────────┐ │ │
│ │ │ windowEntries[] │ │ consecutiveErrors │ │ │
│ │ │ {timestamp,tokens} │ │ (exponential backoff) │ │ │
│ │ └─────────────────────┘ └───────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────┘ │
│ │
│ ctx.on('llm/stream', ...) ctx.on('agent/request-error')│
│ ┌─────────────────────────┐ ┌─────────────────────────┐ │
│ │ 1. Check sliding window │ │ 1. Detect rate-limit │ │
│ │ 2. Delay if needed │ │ 2. Return {kind:'retry'}│ │
│ │ 3. Count output tokens │ │ 3. Record error │ │
│ │ 4. Update window │ └─────────────────────────┘ │
│ └─────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Dependencies
@deepseek-ai/schemastery — configuration schema validation@deepseek-ai/cordis — plugin framework@deepseek-ai/dsh-invariants — DSH invariants@deepseek-ai/dsh-llm — LLM error types
License
MIT
