dsh-web-search-aggregation
中文 · npm · GitHub
An aggregated web-search provider for DeepSeek Harness (DSH): the built-in web_search tool is served through one prioritized queue over AnySearch / TinyFish / Tavily / Brave / Exa / Firecrawl / Jina / SerpApi / Serper — each provider can hold a pool of API keys that rotate per request, every failed attempt falls through to the next provider or key, and the first success wins.
Features
- Prioritized queue, ordered fallback — entries are tried top-down per request; the first one that returns wins, and a failed entry falls through to the next. Order is edited live in the settings card; each provider kind can be queued at most once.
- Multi-key pools with rotation — every provider reads exactly one credential (
ANYSEARCH_API_KEY / TINYFISH_API_KEY / TAVILY_API_KEY / BRAVE_SEARCH_API_KEY / EXA_API_KEY / FIRECRAWL_API_KEY / JINA_API_KEY / SERPAPI_API_KEY / SERPER_API_KEY) whose value holds all of that provider's keys joined by ,. Within one entry the keys are tried in rotating order (round-robin per successful request), spreading load and quota across the pool.
- Works out of the box — AnySearch allows anonymous access, so the queue answers searches before any key is configured. The shipped defaults simply enable all nine kinds — the order is whatever you arrange in the card; the key-required entries fall through until their credential is set.
- Budget-aware — each attempt gets its own deadline (default 10 s, 1–60 s), so one hung upstream cannot eat the tool-level budget; five to six full fallbacks still fit inside 60 s.
- Transparent failures — when every attempt fails, the error reports each one (
[2] tavily/TAVILY_API_KEY#1: 401 unauthorized; …). An empty queue raises WEB_PROVIDER_UNAVAILABLE; caller cancellation raises WEB_ABORTED immediately.
- Secret hygiene — key literals are never logged and never appear in failure records (which cite
REF / REF#N labels only); the settings card shows keys as masked tags and never echoes stored values back.
- Live configuration — the settings card (设置 → 插件 → 插件配置 → 聚合网页搜索 / Aggregated web search) edits the queue, keys, endpoints, and timeout; a committed change reaches the next search without a restart.
How it works
| Half | Location | Responsibility |
|---|
| Host (server) | src/ | Registers the search provider (id aggregated) into ctx.web; cordis.patch.yml pins the web seam's searchProvider to it and inserts the plugin row carrying the default queue. |
| Browser (client) | src/client/ | Registers the Aggregated web search configuration card, which stages the queue and writes changes through the settings + credentials services. |
web_search (tool-web)
└─ ctx.web.searchProvider = aggregated
├─ queue (top-down, first success wins — order user-arranged):
│ anysearch · firecrawl · tavily · tinyfish · brave · exa · jina · serpapi · serper
│ │ keys = credential split on ',' ← pool per provider
│ │ anonymous attempt allowed for key-less AnySearch
│ └─ rotation cursor per entry (resets when the endpoint changes)
├─ per attempt: adapter request under its own deadline (default 10 s)
└─ all failed → WEB_PROVIDER_ERROR with a per-attempt summary
Nine adapters ship today; another upstream is one adapter module plus one registry row.
| Kind | Auth | Default endpoint | Notes |
|---|
anysearch | Bearer, optional | https://api.anysearch.com/v1/search | anonymous access allowed; data.results[] envelope |
tavily | Bearer | https://api.tavily.com/search | sends max_results; the generated answer rides content |
tinyfish | X-API-Key | https://api.search.tinyfish.ai | GET ?query=…; no count control, the seam's maxResults truncates |
brave | X-Subscription-Token | https://api.search.brave.com/res/v1/web/search | GET ?q=…&count=… (count clamped to 1–20, decorations off); page_age → publishedAt |
exa | Bearer | https://api.exa.ai/search | sends numResults (1–100) and contents.highlights; the first highlight is the snippet |
firecrawl | Bearer | https://api.firecrawl.dev/v2/search | sends limit (1–100); no scrapeOptions — plain results, no per-page scrape cost |
jina | Bearer | https://s.jina.ai | POST / with {"q"}; X-Respond-With: no-content keeps it SERP-only (no per-page fetch); num (1–20) only when the request carries a count; description → snippet, publishedTime → publishedAt; EU mirror via https://eu.s.jina.ai |
serpapi | api_key query param | https://serpapi.com/search.json | GET ?engine=google&q=…&api_key=… — the key rides the URL because the API rejects it in headers/bodies; num (1–100) only when the request carries a count (num-bearing calls are documented as more CAPTCHA-prone); snippet → snippet, date → ; a body whose only payload is a top-level string — even at HTTP 200 — fails the attempt so the queue falls through |
Requirements
- DSH web profile (
dsh web), Node.js ≥ 22.19 — the same floor as DSH itself (^22.19.0 || >=24). Source installs from GitHub run the prepare build, which needs the same range.
- At least one reachable upstream: the default queue works with no credentials (AnySearch anonymous); Tavily / TinyFish / Brave / Exa / Firecrawl / Jina / SerpApi / Serper entries need their API keys to contribute.
Installation
From the npm registry (prebuilt — no build permission needed):
dsh plugin --profile web add dsh-web-search-aggregation
From a GitHub repository (source — pnpm runs the prepare build; allowlist the package in profiles/web/pnpm-workspace.yaml if pnpm blocks the build script):
dsh plugin --profile web add github:chendefine/dsh-web-search-aggregation
Or through the DSH plugin marketplace (设置 → DSH插件市场) — the repo carries the dsh-plugin topic and is indexed automatically.
After a bundle plugin is added to the profile layer stack, restart dsh web for it to load; uninstall with dsh plugin --profile web remove dsh-web-search-aggregation and restart again.
Configuration
The settings card (设置 → 插件 → 插件配置 → Aggregated web search) edits the web-search-aggregation settings section live:
plugin configuration card
| Field | Default | Description |
|---|
providers | nine enabled entries — one per kind | The prioritized queue. Each entry: provider kind (each kind at most once), enabled toggle (a disabled entry stays configured but is skipped), and an optional endpoint base URL overriding the adapter's default. The shipped order carries no meaning; arrange it however you like. |
attemptTimeoutMs | 10000 | Per-attempt deadline in ms (1000–60000). One attempt is cut off after this long and the queue moves to the next key or entry. |
API keys are managed on each entry as masked tags: add one key at a time (+ or Enter), reorder by dragging tags off/on — tag order is the order a save writes and the runtime reads. Stored keys are never read back; a save replaces the whole pool, and closing every tag before saving clears the credential. Each provider's keys live in one fixed credential:
| Credential | Provider | Required | Value format |
|---|
ANYSEARCH_API_KEY | AnySearch | no — anonymous access works | one key bare, or several joined by , |
TAVILY_API_KEY | Tavily | yes, for the entry to serve | same pool format |
TINYFISH_API_KEY | TinyFish | yes, for the entry to serve | same pool format |
BRAVE_SEARCH_API_KEY | Brave Search | yes, for the entry to serve | same pool format |
EXA_API_KEY | Exa | yes, for the entry to serve | same pool format |
FIRECRAWL_API_KEY | Firecrawl | yes, for the entry to serve | same pool format |
JINA_API_KEY | Jina Search | yes — the Search API rejects anonymous calls | same pool format |
SERPAPI_API_KEY | SerpApi | yes — the API rejects keyless calls | same pool format (64-hex keys, no prefix) |
SERPER_API_KEY | Serper | yes — the API rejects keyless calls (403) | same pool format (keys carry no prefix) |
Combining with other provider plugins
A bundle patch replaces the web seam row's whole config, so the layer applied last determines the final searchProvider / fetchProvider pair. This plugin's patch states only searchProvider: aggregated — it owns the search selection and nothing else. Two consequences:
-
With dsh-web-fetch-playwright installed before this plugin (or without it entirely), nothing needs adjusting: an unconfigured fetchProvider resolves to the single registered fetch provider automatically.
-
If another plugin's layer is applied after this one and resets searchProvider, pin the combined selection in your profile's own profiles/web/cordis.patch.yml — the user layer wins over every bundle layer:
- id: web
config:
searchProvider: aggregated
fetchProvider: playwright
Development
pnpm install
pnpm typecheck # tsc --noEmit
pnpm test # vitest run (103 unit tests, no network)
pnpm build # tsc declarations + tsdown (host ESM + client module-registration bundle)
Repository layout:
src/
├── index.ts # host entry: registers provider + settings section
├── config.ts # schemastery schema, defaults, queue normalization
├── provider.ts # AggregatedSearchProvider: queue walk, rotation, deadlines
├── keys.ts # ','-joined key-pool vocabulary (parse/format/mask)
├── types.ts # queue entry, config, per-attempt failure records
├── adapters/ # AnySearch / Tavily / TinyFish / Brave / Exa / Firecrawl / Jina / SerpApi / Serper adapters + shared HTTP
└── client/ # browser half: settings card, form model, locales
tests/ # config, keys/adapters, provider, client-controller
See CONTRIBUTING.md for the development and release workflow, and SECURITY.md for the security model and reporting policy.
Security
Search queries and API keys leave the machine only toward the endpoints configured per queue entry (the nine providers' official APIs by default). Keys are stored in the DSH credentials domain — never in the settings file, never logged, never echoed back to the client. Failure records and logs cite masked references (TAVILY_API_KEY#2), not literals. No result data is retained beyond the session that requested it.
License
MIT © 2026 chendefine
