@mi-lu-uncle/dsh-pelican-watch
English | 简体中文 | 日本語
A DeepSeek Harness community plugin that runs a long-term
model-quality probe (降智监测) on a schedule, across any number of
models:
- Pick your models — the gear in the panel's top-right corner lists every model the
configured providers report (live
/models discovery, merged with the models
in your DSH profile) and probes a multi-selected set, each model through its
own provider and API key.
- Scheduled drawing — every 30 minutes (configurable) each selected model is
asked to draw a pelican riding a bicycle as a single-file HTML page with an
inline SVG.
- No skills, no tools — each run is a bare
chat/completions request with
no tool definitions and no agent loop, so the result measures the model
alone, never your local skills.
- Everything flat, nothing to click — the main board is read-only: one band
per model, its status and figures on the left (when its newest drawing landed,
runs, average duration, how many came back without a drawing, and the failure
reason) and every drawing it ever made on the right, all laid out at once —
nothing collapsed, filtered, or hidden behind a click.
- Run state in the header — while probing: “n/m done”, which models are in
flight, how long the slowest has waited (m:ss) and the timeout cap (m:ss), so a
slow model and a stuck one are told apart at a glance.
- One gear button — the panel's own top-right corner opens a settings sheet:
pick models, probe now, open one drawing's original page. The board below stays
pure viewing, so nothing has to be clicked to be read.
- Stop it, retune it — that same sheet can pause / resume automatic
probing, change the interval (1–1440 minutes), and stop the pass that is
running right now: requests in flight are aborted, models that had not started
are skipped without a request being sent. Pausing is written to
settings.json immediately and survives a restart, so a plugin you only probe
by hand never costs a token on its own.
- Bounded history — keeps the newest N drawings (12 by default), pruning
older ones automatically. Same prompt, same model, over days: degradation
shows up as worse structure, refusals (saved and badged 「未检出 SVG」), or
latency growth.
Install
From this GitHub repository (recommended):
dsh plugin --profile desktop add https://github.com/mi-lu-uncle/dsh-pelican-watch.git
Or from a local path or tarball:
dsh plugin --profile desktop add /absolute/path/to/dsh-pelican-watch
Note: link:-installed plugins are not watched by HMR, so editing
lib/index.js or lib/client.js requires an app restart to take effect.
Choosing models
The gear in the panel's top-right corner opens the settings sheet: every provider found in your DSH profiles, with
the models it serves (its own /models endpoint, merged with whatever the
profile already lists). Tick any number of models and Save models — from the next
cycle on each selected model draws its own picture and gets its own band on the
board.
That selection lives in dir/settings.json and is what the scheduler uses;
config.model / config.targets only seed the first run. The same file holds
two knobs that decide what you spend, both changeable from the panel's gear
without touching any config file:
| key | default | meaning |
|---|
autoEnabled | true | Off disarms the timer at once (nextRunAt goes empty); only the manual “probe now” then spends anything, and the choice survives a restart. |
intervalMinutes | follows config.intervalMinutes (30) | Changing it in the panel re-arms the countdown immediately — no restart. |
config.model / config.targets only seed the very first run. At most 12 | | |
models are probed per cycle — concurrency of them at the same time (3 by | | |
| default), one drawing each. | | |
Speed and timeouts
- Models run in parallel (3 at a time by default), so one slow model no
longer delays the others: every drawing is written to disk the moment it
arrives and shows up in the panel within seconds.
- Each request has its own timeout (
timeoutSeconds, default 180). A model
that exceeds it is reported as 超过 N 秒没有返回 on its own band instead of
hanging the whole pass. Reasoning models can genuinely be slow — measured
against one provider: qwen3.7-max ~52s, deepseek-flash ~121s (15k of those
completion tokens were reasoning), while glm-5.3-flash sometimes exceeded
4 minutes.
- A timeout or HTTP error is not a drawing: nothing is saved and it does not
count as a drawing, because that is an infrastructure fact rather than model
behaviour. A model that answers without an SVG (a refusal) is saved and
marked
没出图 — that is the signal worth looking at.
maxTokens is not sent by default: reasoning models spend tens of thousands of
completion tokens before they draw, and a tight ceiling would truncate the SVG
instead of speeding anything up.
Configuration
Defaults match the usual profile setup; override any of them on a
pelican-watch row in your profile patch (cordis.patch.yml):
- id: pelican-watch
config:
model: glm-5.3-flash # seed model (the panel's selection wins)
baseURL: https://taotoken.net/api/v1 # OpenAI-compatible endpoint
apiKeyEnv: TAOTOKEN_API_KEY # env var holding the API key…
# apiKey: sk-… # …or put the key here directly
# targets: # optional: seed several models at once
# - { provider: taotoken, baseURL: https://taotoken.net/api/v1, apiKeyEnv: TAOTOKEN_API_KEY, model: glm-5.3-flash }
intervalMinutes: 30 # drawing cadence
maxImages: 12 # history depth (per model ≈6h at 30min)
timeoutSeconds: 180 # per-request ceiling; raise it for slow models
concurrency: 3 # how many models are probed at once
# maxTokens: 8192 # optional; unset by default (see below)
dir: ~/.dsh/pelican-watch # where drawings are saved
runOnStart: true # draw once ~8s after boot
# temperature: 0.7 # optional; omitted by default
# prompt: … # only if you must change the benchmark
The benchmark prompt is deliberately fixed — changing it makes the history
incomparable, so treat prompt as a last resort.
API key
The key is resolved at call time, in this order:
- a literal
apiKey in config;
process.env[apiKeyEnv] (default TAOTOKEN_API_KEY);- the same name in DSH's own credential store —
refs: in
~/.dsh/.credentials.yaml (override the path with credentialsFile).
Step 3 is why a DSH-configured provider key works with no extra setup: the
desktop app is launched from Finder/Dock and therefore does not inherit
environment variables exported in a shell, while refs: already holds the key
under the provider's apiKeyEnv name.
With none of the three the run is skipped and the panel shows the reason
(缺少 API Key：…) — set any one of them and the next scheduled run proceeds.
Display only — no scoring
This plugin does not score anything. It does two things: it asks every model
the same drawing prompt on a schedule, and it lays the drawings out as they are.
Earlier versions computed a 0-100 "structural score" offline (is there an SVG, two
wheels, a frame, wings…). That has been removed on request: such a number can
only approximate “did these elements appear”, and it lends a false precision.
The judgement is left to whoever looks at the pictures (or to a vision model):
same prompt, same model, over consecutive days — the drawings themselves are
the finding, and no digit has to stand in for them.
The numbers that remain on the board are not grades, just facts: how many runs,
average duration, when the newest drawing landed, and how many runs came back
without a drawing. 没有出图 means the request returned but carried no usable
SVG (typically a refusal); the raw answer is still saved so you can read it.
Storage
Drawings live in dir (default ~/.dsh/pelican-watch):
| File | Role |
|---|
<时间>_<模型>.html | The full generated page; first line is a meta comment (model, provider, time, duration, whether an SVG was detected) |
<时间>_<模型>.svg | The extracted inline SVG, used for the panel's <img> preview |
settings.json | The selected probe targets (written by the picker) plus the two pace knobs autoEnabled / intervalMinutes |
Deleting files there is safe; the panel always lists what is on disk.
Privacy & safety
- Every route is same-origin fenced like other plugin surfaces.
- Generated pages are served as
text/plain and rendered inside a
sandbox-less iframe (sandbox=""), so model output can never execute
scripts in the app origin; SVG previews are served with
x-content-type-options: nosniff.
Layout
| File | Role |
|---|
lib/index.js | Host half: multi-model scheduler, storage, scoring, /api/pelican-watch/* routes |
lib/client.js | Client half: the flat board (one band per model with all its drawings) plus the gear-opened settings sheet (React) |
cordis.patch.yml | Bundle patch that mounts the package as one host row |
package.json | npm metadata + dsh.bundle/dsh.client/dsh.compatibility |
test/pelican.test.mjs | Offline test: per-model roll-up, provider parsing, target selection, schedule control, route wiring |
Test
node test/pelican.test.mjs
