dsh-jev-verify

Jev Verify

面向 DeepSeek Harness 的 Jev（TypeSafe System One 决策模型）——七种工具：jev_decision（在一次并行调用中处理最多 25 个类型化问题，测得中位数为 266-484 毫秒）、jev_choose（对 2-10 种方案进行排名）、jev_batch（对 1-20 个 tex 进行一组最多 5 项判断）

目标 Profile

插件会安装到这里；不确定时保持 web。

npx -y @deepseek-ai/dsh plugin --profile web add dsh-jev-verify@0.8.6

兼容性与来源证明

Jev Verify 以 dsh-jev-verify 发布，当前版本为 0.8.6。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。

DSH 兼容范围: *
运行环境: web
发布来源: npm
Registry 更新时间: 2026/10/16

版本

0.8.6stable

2026/10/7

0.8.5stable

2026/10/7

0.8.4stable

2026/10/7

查看其余 13 个版本收起版本

0.8.3stable

2026/10/7

0.8.2stable

2026/10/6

0.8.1stable

2026/10/6

0.8.0stable

2026/10/6

0.7.5stable

2026/10/4

0.7.4stable

2026/9/30

0.7.3stable

2026/9/30

0.7.2stable

2026/9/28

0.7.1stable

2026/9/28

0.7.0stable

2026/9/28

0.6.0stable

2026/9/26

0.5.0stable

2026/9/24

0.2.0stable

2026/9/21

README

dsh-jev-verify

Jev — TypeSafe AI's System One decision model — as a first-class plugin for DeepSeek Harness (dsh).

Jev does not generate text. Given a state plus typed questions it returns typed answers with calibrated probabilities in one parallel API call (~70–500 ms published; in our runs median 266–484 ms and p95 825–1468 ms; single-run maxima reach ~1.5 s and ~5 s under a loaded network). This plugin exposes that as agent tools, adds an opt-in auto-guard (risk + loop checks), an honest local usage/quota panel, and makes sure the claims are verified, not trusted blindly:

Tool	What it does	Call it when
`jev_decision`	Up to 25 typed questions (`choice` / `score` / `noul`) about one `state`, answered in parallel in one call (published ~70–500 ms; measured median 266–484 ms and p95 825–1468 ms across our runs, with single-run maxima of ~1.5–5 s depending on network load). Every answer carries a calibrated confidence and probabilities; the result also reports model, latency, token usage and estimated cost.	You need fast, repeatable verdicts instead of prose: classification/labeling, routing or triage, priority/severity/satisfaction scoring, spam/toxicity/PII checks, intent or truth checks, extracting structured tags. Batch related judgments into one call — parallel, no extra latency.
`jev_choose`	Ranks 2–10 candidate options/approaches: each gets a fit score 0–3 and a risk noul, combined into a composite `fit/3 × (1−risk)`; returns an ordered table, a recommended pick, and per-option latency/cost.	Several independent approaches are on the table at a fork and you want a calibrated tiebreaker before deciding. Jev scores, it never explains — the reasoning stays yours.
`jev_batch`	Runs one set of 1–5 judgments over 1–20 texts: one real call per text (4-way concurrency by default, 6 max), returning a paste-ready markdown table sortable by a judged dimension; built-in `sentiment` / `priority` / `intent` / `pii` / `spam` presets, or your own questions in the `jev_decision` shape.	A batch of tickets/comments/logs/feedback needs labeling, classification, prioritization or PII/spam screening, or several candidates need ranking on one dimension — the tool for "many texts, same judgments"; never hand-roll one `jev_decision` per item. Each text is its own real call, and a failed row says 失败 with the raw error instead of guessing.
`jev_verify`	Runs the frozen 27-question labeled benchmark (urgency, spam, toxicity, personal data, routing, intent, search type, priority, severity, satisfaction, guard verdicts) against the live API at concurrency 6 (measured 4.96 s wall for all 27 calls in 0.7.5 and 3.999 s on a re-run, versus 10.1 s serial): accuracy overall and on the high-confidence subset, median/p95/min/max latency, calibration, tokens, cost, and every mislabeled case (total plus the high-confidence ones).	Endpoint health check, model-version comparison, or a regression check — never routinely: one run is 27 real API calls (~8.7K input tokens, ≈$0.0004).
`jev_guard_status`	Read-only audit of the auto-guard: deterministic rules and the Jev backstop counted separately (`checks` / `jevCalls` / `denied` / `deterministicDenied` / `auditCalls`), guarded tool names, `denyThreshold`, loop-check counts, remaining session budget.	Confirm the guard is armed, see how often it actually fired, or explain why a command was blocked. When the guard is off it says so instead of reporting zeros.
`jev_overview`	Read-only snapshot of this session's Jev ledger: recent decisions and choices with confidence, latency median & p95, question-type mix, guard events, cumulative tokens and cost, plus key/guard/threshold status.	The user asks what Jev has done, blocked or spent — or you need endpoint and budget state without restarting the session. The ledger starts at plugin-instance lifetime; guard counts live in `jev_guard_status`.
`jev_usage`	Locally measured Jev usage and (optional) local budgets: rolling windows (today / 7 d / 30 d / all / session), calls, input/output tokens, estimated cost, latency median & p95, question-type and per-tool mix, a projected daily burn rate, and — with `quota.enabled` + `quota.enforce` — a hard stop before the API call.	The user asks how much Jev has been used or what it cost, or you want to check a budget before a batch of judgments. Reads only local state — no API call. TypeSafe has no balance endpoint, so this is measured local usage, not provider credit.

Auto-guard mode (opt-in autoGuard.enabled): before shell-like tool calls (bash/pwsh/run_code/terminal), a free deterministic layer runs first. Seven hard rules block catastrophic, irreversible operations — recursive deletion of a filesystem, drive or directory tree, disk formatting, database destruction, credential exfiltration, force-pushed git history — and three soft rules (host restart / power-off, git-history rewrites short of a force push, always-true DELETE/UPDATE conditions) are handed to Jev (risk noul ≥ threshold ⇒ deny). Hard patterns only fire in executable position, so quoting or documenting a dangerous command is never blocked, and a hard match found elsewhere is downgraded to a Jev-judged hint. Jev failure is fail-open with a warning. A loop guard evaluates repeated same-tool calls for semantic stalls and injects advice instead of blocking. Every verdict is recorded in the session ledger and auditable via jev_guard_status.

Honest by design — no mock mode, no silent fallback:

without a TYPESAFE_API_KEY, tools and guard fail with explicit setup instructions (the guard fails open with a warning — it never silently pretends to have checked);
every jev_decision result includes the model, latency and token usage, so each call is auditable;
jev_verify refuses to report numbers it did not measure;
the benchmark CLI (bench/bench.mjs) is dependency-free and reproducible with any key.

What's new in 0.8.6

Fixed a dependency mistake that could stop the host from starting (issue #1): @deepseek-ai/dsh-tools was declared as a normal dependency. In a profile using nodeLinker: hoisted + hoistPattern: ["*"], pnpm materialised a real copy under the profile's node_modules/@deepseek-ai/; the tools loader entry resolved to that copy instead of the host's own, it failed to import its peers, and startup ended in 1 required plugin did not activate. It is now an optional peer (^0.1.5-rc.2) — never installed, so it can never shadow the host. dependencies now carries only the leaf library @deepseek-ai/schemastery.
New guard test test/manifest.test.mjs fails if any host runtime package reappears in dependencies (verified by reintroducing the bug and watching it fail). Suite: 38/38.
Otherwise identical to 0.8.5.

What's new in 0.8.5

"Remaining" finally has an honest source — the balance you declare, minus locally measured spend — instead of a number something made up.

A restart no longer zeroes the total. quota.persist now defaults to true: the ledger is written to $DSH_HOME/jev-usage.json, falling back to ~/.dsh/jev-usage.json when the host leaves DSH_HOME empty (that missing fallback was the root cause of the cumulative total always reading 0).
Remaining = declared − measured. TypeSafe exposes no balance endpoint at all (/v1/usage, /v1/quota, /v1/account, /v1/balance, /v1/credits, /v1/billing, /v1/me, /v1/limits, /v1/wallet, even /v1/openapi.json all 404, and /v1/models carries no quota/credit headers), so the provider side cannot be read. Set quota.declaredBalanceUsd and quota.balanceSince (e.g. 2026-10-01) and the panel prints 剩余 = 自报 $5.00 - 本机实测 $0.000004（2026-10-01 起）; leave them empty and it says the balance was not declared rather than guessing one.
The pill re-reads every 20 s (was 60 s), so a call you just made shows up while you are still looking at it.
The collapsed pill leads with 剩余 once a balance is declared (Jev · 剩余 $4.9997 · 今日 3 次), and the popover renames the cumulative row when the ledger is on disk.
New jev_batch — one call, one set of judgments, over a batch of texts, returned as a table. 1–20 items × 1–5 questions, one real Jev call per item (4-way concurrency by default, 6 max), returning a paste-ready markdown table; sort=auto|desc|asc|none orders rows by a judged dimension (auto only when the single question is a score, descending). Built-in sentiment / priority / intent / pii / spam presets, or your own questions in the jev_decision shape. A failed row is marked 失败 with the raw error — never guessed — and every row is metered separately (kind: "batch"), so N items really are N calls and N costs.
The settings card gained the two declaration fields; npm test is 37/37 (the new test/batch.test.mjs adds 7 cases: one real call per item with measured cost, the three sort modes, an unguessed failed row, a whole-batch failure that throws, preset expansion with the shared context prefix, 11 rejected argument shapes that spend no call, and the concurrency cap).

What's new in 0.8.4

The panel stops implying a quota that does not exist, the guard's own Jev calls finally reach the ledger, and the plugin now participates on its own.

No fabricated quota. TypeSafe is a metered API with no balance endpoint. With no local budget configured, the pill and the status page no longer print a "额度正常" verdict, no longer draw percentage bars against "∞", and no longer show "额度重置 13 小时 41 分后" — a countdown to a budget that was never set. Without a budget the panel states one thing: pay-as-you-go, nothing resets, only locally measured usage is reported. A "本地预算重置" countdown appears only when a local limit really exists, and it is labelled local.
Money first. The collapsed pill reads Jev · 今日 $0.000534 · 29 次, and the popover leads with today's cost and cumulative cost; call counts are secondary rows. Cost is what a pay-as-you-go user actually pays.
The guard's Jev calls are metered. judgeRisky/judgeStall used to answer the guard and vanish: a session could report safety.jevCalls: 2 in jev_guard_status while the usage ledger still read "今日调用 0 次". Both verdicts now report latency and token usage through an onJevCall event and are recorded as kind: "judge" ledger entries (tool: "auto-guard").
Auto-triage (autoTriage, on by default). On the first step of every turn one small Jev call (4 routing questions, ~1.5K input tokens, bounded by autoTriage.timeoutMs) classifies the request — intent, whether an atomic judgment is really needed, which kind, and which Jev tool serves it — and the advice is injected back as a plugin-sourced user message. It runs at most once per (agent, turn), never blocks a step, and fails open on an unknown host shape, a missing key, a timeout or any API error. Being metered, it is visible in the panel, so a working plugin no longer shows zero calls.
Harder guidance. The system-prompt section now carries mandatory-trigger rules (classification/routing, priority/severity/satisfaction scoring, truth and compliance checks, structured extraction, choosing among 2-10 options), forbids bypassing Jev "because I already know the answer", and states the real cost magnitude (~$0.00002-0.0001 per call; 25 questions cost the same as 1).
Tests: new test/triage.test.mjs plus extended assertions in test/usage.test.mjs, test/client.test.mjs and test/guard.test.mjs; npm test is 30/30. The client half takes effect on a page refresh; the server half (guard metering, auto-triage, guidance, status page) needs a host restart.

What's new in 0.8.3

Settings you save now actually take effect.

Root cause. The card handed dotted field names to scope.set(field), and that call stores path: [field] — one segment. A field such as dashboard.basePath was therefore written into the settings document as a single literal key named dashboard.basePath, which no schema ever resolves. Saving reported success and changed nothing: on a live host ~/.dsh/settings.yaml held the flat keys autoGuard.maxJevCallsPerSession: 60 and dashboard.basePath: jev while the resolved document still showed the schema defaults (50 and /jev). Found by reading the live document over the settings/describe RPC after the 0.8.2 restart.
Fix. The card builds { op: "set", path: [<segment>, …] } operations and commits them in one atomic scope.mutate(ops) call; it only falls back to chained scope.set on hosts that expose no mutate.
Repair on open. A one-time mount effect rewrites documents written by <= 0.8.2 that contain literal dotted keys: for each one, set the nested path and unset the literal key, in one atomic mutation, guarded by a WeakSet so it runs once per scope and wrapped so a failed repair can never break the card.
Client-only: refresh the page — no host restart. test/client.test.mjs PASS 7 covers path splitting, the generated operations, the atomic repair and its idempotence. npm test is 29/29.

What's new in 0.8.2

Two things the board got wrong, plus a usage surface that does not need the board at all.

/jev never 404s again. dashboard.basePath used to be taken verbatim, so a value such as jev (no leading slash) registered the literal path jev, which no browser request ever matches. Paths are now normalized (leading slash added, trailing slashes trimmed, empty or / falls back to /jev) and both /jev and /jev/ are registered. With dashboard.enabled: false the path still answers with a live status page — today's calls, cost, this instance, today's tokens and latency, quota state — that points at the composer pill and says how to turn the full board on, instead of a bare host 404.
/jev/api/usage — a small read-only JSON route that always mounts (it does not wait for dashboard.enabled) and projects the same locally measured snapshot the jev_usage tool returns.
A composer usage pill (conversation.input.right, beside the submit button), modeled on dsh-opencode-go: Jev · 今日 29 次 · $0.000534 with a warn/danger tone dot, expanding into a 270 px popover — today's calls and cost against their limits, this instance's calls, today's tokens, median and p95 latency, guard denials, the top three tools, time to reset, last refresh and a manual refresh. It polls every 60 s and on tab focus, falls back to the full board API when the pill route is absent, and degrades to Jev · 额度不可用 carrying the exact error — it never throws. It reports local measurements only, never a vendor balance; jev_usage in the conversation stays the full panel.
Regression tests: test/dashboard.test.mjs covers the path normalizer, the route table with the board on, off and on a custom basePath, the 200 status page and a failing snapshot; test/client.test.mjs covers the pill payload's three shapes, the label and tone states, and the rendered popover text. npm test is 29/29.

What's new in 0.8.1

Fixes the Settings > Plugins card that showed 未配置 / unconfigured and blank fields while a key was configured.

Root cause: the schema registered with the settings service marked the GUI fields with the volatile marker. On a host whose schemastery understands it (3.18.4), resolve() replaces every marked field with a live reference object, and the settings service publishes registration.resolved verbatim — so the wire value was {"enabled":{},"apiKey":{},"apiKeyEnv":{},"quota":{"enabled":{},…}}. The GUI ships its own older schemastery (3.18.2), which cannot decode that ($.enabled expected boolean but got [object Object]), so the card silently fell back to an empty draft: badge "未配置", every field blank. The plugin's own reads were unaffected (it unwraps those references), which is why judgments, the guard and jev_usage kept working throughout.

One field definition, two schemas. buildConfig() builds both Config (entry form; keeps the marker for hosts that understand it) and SettingsConfig (plain — the one registered with the settings service), so the 23 fields cannot drift apart.
The card no longer trusts a single decode path. When the decoded value is empty it reads the raw base/user layers the service already sends, says so on the card ("服务端配置值未能解码…"), and stays editable and savable.
A credential reference is not "configured". The badge has three honest states: literal key / 环境变量 <NAME> (unverified) / none.
Regression tests: test/config.test.mjs asserts the registered schema is plain and JSON-safe on the wire; test/client.test.mjs renders the card against an undecodable snapshot and asserts the badge names the configured reference and never says 未配置. npm test is 29/29.

Reproduced on the wire (deployed plugin, dsh 0.1.5-rc.2, 2026-10-06):

schema	empty-object fields on the wire	GUI decode
`Config` (entry-form only)	8 + every `quota.*`	FAILS — `$.enabled expected boolean but got [object Object]`
`SettingsConfig` (registered)	none	passes; `apiKeyEnv="TYPESAFE_API_KEY"`, `enabled=true`, `autoGuard.denyThreshold=0.8`

What's new in 0.8.0

The usage/quota panel: Jev usage is now measured, stored and surfaced — as a tool view, in the overview card, and on the standalone board.

jev_usage — a usage/quota panel you can call. Every decision, choice, verification and guard event is recorded locally with its measured input tokens, cost, latency, question-type mix and tool name. The tool returns rolling windows (today / 7 days / 30 days / all / session), a daily call and cost series, a projected daily burn rate ("at this pace the daily limit is reached in N h"), and an optional hard stop.
Local budgets, not invented provider balances. TypeSafe publishes no balance or quota endpoint (GET /v1/usage, /v1/quota, /v1/account, /v1/balance, /v1/credits, /v1/limits, /v1/billing … all 404; only /v1/models answers). So the panel reports only what this machine actually measured, against limits you set yourself: quota.dailyCallLimit, quota.dailyCostLimitUsd, quota.sessionCallLimit, plus warnAtPercent. The boundary is printed in both the text result and the card — never a fabricated balance.
Optional hard stop (quota.enforce). With enabled + enforce, a call that would cross a configured limit is refused before the API request, with a message naming the budget. Display-only by default.
Persistence (on by default since 0.8.5; opt-in in 0.8.0–0.8.4). 0.8.0–0.8.4 kept everything in memory (400 samples) unless you set quota.persist: true; since 0.8.5 up to quota.historyDays (default 30) of daily history is written to $DSH_HOME/jev-usage.json by default (atomic tmp+rename, fail-open), so the cumulative total survives a restart.
Surfaced everywhere. jev_overview gained a usage block, the standalone board (dashboard.enabled) gained a quota section with bars, projections and a 30-day sparkline, the settings card gained a "usage/quota" group, and jev_verify now records its 27 calls, tokens and cost instead of only the accuracy string.
Tests: 28 passing — the new test/usage.test.mjs covers the empty state, token accounting, batch calls, quota status and the hard stop, persistence across a day boundary, reload, reset and every formatter.

What's new in 0.7.5

The audited release: a three-way review (server, client, packaging) turned up a batch of correctness and honesty bugs, all fixed and pinned by tests — and jev_verify now runs its 27 cases concurrently.

Server

The session ledger records even when the dashboard is off. record() used to return early unless the standalone page was mounted, so jev_overview reported 0 calls / 0 tokens / $0 forever while jev_guard_status counted the same events. The in-memory ring now always records; only the HTTP page and the $DSH_HOME/jev-roll.jsonl append stay behind dashboard.enabled (this README and the setting text now say exactly that).
The guard reports the events it already knew about. The Jev denial path now emits onSafetyDeny (with confidence and the tool name) and the loop path emits onLoopAdvisory, so the dashboard advisory counter and the Jev-verdict branch of the ledger are no longer dead code.
Auto-guard rebuilt in two deterministic tiers with position awareness (see Auto-guard below): scanning a whole argument list with regular expressions is gone; the root-wipe finder walks flag tokens one by one, so extra flags, --flag=value forms and reversed flag order can no longer slip a root wipe past; a recursive delete of an ordinary relative directory (for example a build output folder) is a Jev-judged soft hint instead of a hard block; and a hard pattern that only appears in quoted prose is downgraded the same way.
TYPESAFE_BASE_URL / TYPESAFE_MODEL overrides work. The schema defaults were being read as if they were user settings, so the environment variables never won and self-hosted endpoints were silently sent to the official base URL.
jev_verify runs in batches of 6 (VERIFY_CONCURRENCY): measured 4.96 s (3.999 s on a re-run) wall for the same 27 calls that took 10.1 s serially, and it no longer flirts with the tool own timeout.
mislabeled is complete again and a new mislabeledHighConfidence lists which misses were high-confidence (previously one high-confidence miss hid every low-confidence one).
Errors name the tool that failed. missingKeyError() and the abort / timeout / HTTP messages were hardcoded to jev_decision even when jev_choose or the playground raised them.
Registration failures surface instead of scrolling past. safe() now accumulates regFailures, and jev_guard_status returns them as registrations.failures / failureCount — the failure class that shipped two broken releases.
(), and the Jev verdict cache is bounded at 200 entries.

Client (in-chat views and the settings card)

noul confidence is no longer inverted. A "no" verdict with noul: 0.02 used to display as "no · confidence 2%" in an alarm colour; it now shows the decision-side confidence (98%), matching the overview board.
The overview card shows average latency and the question-type mix that the presentation projection had been sending all along.
jev_choose is auditable: the tool view keeps the ranked options and the submitted candidate text plus the selection context, instead of only a count.
The running chip is honest: only jev_decision / jev_choose claim "0 questions" / "0 candidates" while a call is in flight; other tools show none.
Dead CSS variables fixed (--dsw-alias-bg-l2 → --dsw-alias-bg-layer-2): the recommendation row and chips were invisible in the dark theme.
The dashboard page reports its own failures instead of swallowing them into a permanent "loading…", and tolerates events without a timestamp.
The credential badge is three-state: a literal key shows "configured", an environment-variable reference shows "environment variable (not yet verified)", and neither shows "not configured" — an env name is no longer mistaken for a working credential.

Packaging and docs

test/functional.mjs → test/functional.test.mjs, so the real HTTP-contract and grading tests actually run under npm test (they never did: the glob only matched *.test.mjs).
bench/choose-e2e.mjs no longer hardcodes a developer home directory and reads the key from TYPESAFE_API_KEY or the DSH env file.
bench/bench.mjs takes --concurrency (default 6) and reports wall time; the user agent is versioned.
README: latency tail percentiles and the test count are stated as measured, engines.dsh is marked advisory, and the file no longer advertises a blacklist that scanned text it had never inspected.
Optional peer dependencies declared (dsh-credentials, dsh-client-locale, dsh-client-ui-settings, dsh-api-remotes).

What's new in 0.7.4

jev_choose finally registers — in 0.7.3 the tool did not exist in any host. Its options array parameter declared minItems/maxItems, keywords the host's value-schema DSL rejects (JsonSchemaError: unsupported JSON schema: parameters.options.minItems is not supported by the value schema DSL). defineTool threw, the per-tool quarantine swallowed the throw, and the release shipped a headline feature that no host could call — the boot test only asserted four tools, so nothing failed. 0.7.4 drops the unsupported keywords (the 2–10 option count, the ≤800-character options and the ≤2000-character context stay enforced at runtime by validateOptions in lib/counsel.js) and test/boot.test.mjs now asserts all five tools register through the real defineTool schema compiler.
The guidance section now actually registers — this is what makes the harness call Jev on its own. Until 0.7.3 the plugin asked the host for a section order via getSectionOrder("TOOL_JEV"), a slot that does not exist in the current DSH generation; the lookup returned undefined and the host's systemPrompt.section() rejects a non-finite order (TypeError: prompt section "<name>" order must be a finite number). The throw was swallowed by the same quarantine that protects the profile, so the prompt section silently never existed and the model only ever saw the tool descriptions. 0.7.4 resolves the order as getSectionOrder("TOOL_JEV") → guidance.order → 3000, logs the outcome (system prompt: guidance registered | section tool:jev | order N | chars M), and test/boot.test.mjs now fails if the registered order is not a finite number.
Sharper functional descriptions (what / when / when not). All five tool descriptions now state the measured latency, the question-type rules, the batch-in-one-call advice, the cost of jev_verify, and an explicit do not use for boundary (jev_decision is verdicts, never prose; jev_choose scores without explaining). The system-prompt guidance is generated from the same wording, so the prompt and the tool list cannot drift apart.
New guidance config group: guidance.enabled (default true), guidance.order (default 3000), guidance.extra (deployment-specific rules appended verbatim) — editable in the GUI settings card.

What's new in 0.7.3

jev_choose — multi-option counsel. New tool that ranks 2–10 candidate approaches in one call: each option is scored by Jev (fit score 0–3 with calibrated confidence + risk noul), combined into a composite (fit/3 × (1−risk)); returns an ordered table, a recommended pick and honest per-option latency/cost. Rendered in-conversation as a ranking table (ChooseBody); its guidance text was intended for the agent system prompt, but that registration silently failed until 0.7.4 (see below), and included in the dashboard/decision-board stats (kind: choose merged into decisions). Caveat: 0.7.3 shipped this tool broken — see the registration fix in 0.7.4 below.
Deployed 0.7.3 to D:\lab\jev (vendor) and the web profile pnpm store with byte-identical MD5 on all five changed files; node --check green; rankOptions unit-verified against a mock transport.

What's new in 0.7.2

Benchmark claims now match the code. jev_verify's tool description said "24 questions across 15+ cases" while lib/cases.js has held 27 labeled questions (27 cases, guard verdicts included) since v0.2.0. The description, README, market PR copy and the verification report now all state 27.
Re-measured on 2026-09-28 (run 4, before this release): 96.3% (26/27) — the same single recorded boundary miss as run 3 — 8,696 input tokens ≈ $0.000365, median 484 ms (network-dependent). Full terminal log: bench/run4.log.

What's new in 0.7.1

Docs sync (0.7.1 ships 0.7.0 code). The verification report now carries the complete measured history instead of only the latest run: the 2026-09-21 benchmark/guard runs, the 2026-09-23 GUI card, settings-card redesign and in-chat tool views, the 2026-09-26 two-generation settings-API compatibility work, and the 2026-09-28 v0.7.0 regression on a headless profile. docs/verification.md is the full 208-line record; no behavioural change versus 0.7.0.
Structured in-chat views for every tool. Tool results now carry a presentationMeta projection (kind: decision | overview | guard | verify), so the conversation renders a decision card, a guard board and a benchmark report — answers, confidence bars, status chips and stat grids — instead of raw text. The view falls back to parsing the tool text when meta is absent.
Full plugin settings card. Settings > Plugins > Plugin configuration > Jev now covers every option: credentials (key, credential-ref, base URL, model, timeout, question cap), tool toggles, the whole auto-guard block (safety/loop switches, guarded tool list, deny threshold, Jev budget, loop tuning) and the dashboard (enable + base path) and the usage/quota panel (enable, enforce, warn threshold, daily call / daily cost / session limits, persistence, history days) — with numeric validation and dirty-state handling.
Config handling hardened. Volatile config fields (schemastery .volatile()) are unwrapped before use, so an object-shaped apiKeyEnv no longer crashes credentialRef(...), autoGuard.enabled / dashboard.enabled actually take effect, and numeric options are read as numbers.
Tests: 28 passing (npm test) covering boot, client render, tool views, settings, dashboard, guard, usage/quota accounting and the presentation projections.

Why Jev

TypeSafe AI (founded by Diogo Almeida, ex-OpenAI / ChatGPT research) released Jev on 2026-09-15 as its first “System One” model: unstructured state in, typed probabilistic decisions out. For agent workloads the model runs 40–200x faster than frontier chat LLMs on System One-shaped tasks ($0.042 per million input tokens, free output). Ideal for the high-frequency fast judgments of an agent loop: classification, routing, triage, scoring, guardrails, truth checks.

Install

Requires Node >= 20 and dsh >= 0.1.5-rc.2. engines.dsh is advisory: npm enforces only the node key, so the real requirement is a host that exposes dsh-tools 0.1.5-rc.2 and a settings service — the plugin degrades one surface at a time (and reports it in jev_guard_status) if one is missing.

dsh plugin --profile web add dsh-jev-verify

then add a loader entry to $DSH_HOME/profiles/<profile>/cordis.patch.yml:

- insert:
    - id: jev-verify
      name: dsh-jev-verify
      config:
        autoGuard:
          enabled: true     # opt-in auto-guard

Restart the profile (or use the plugin market's one-click install, which performs both steps).

Tip: for the dsh market UI install dshmarket first (dsh plugin --profile web add dshmarket) and install this plugin from Settings > 插件市场.

API key

Get a free key at https://console.typesafe.ai/keys. Then choose one:

export TYPESAFE_API_KEY=... in the launching environment, or
Settings > Plugins > Plugin configuration > Jev (credentials service), or
set apiKey in the plugin config.

Optional environment overrides, with real precedence over the schema defaults since 0.7.5: TYPESAFE_BASE_URL (default https://api.typesafe.ai/v1) and TYPESAFE_MODEL (default jev-latest). Before that fix the schema default was read as if it were a user value, so setting these changed nothing.

Usage

Ask the agent to use jev_decision for any fast judgment. Example prompt:

Use jev_decision with state = "" and these questions: department (choice, criteria billing/technical/sales), is_urgent (noul), severity (score, 3 levels). Report answers + confidence.

Or request jev_verify at any time to confirm the endpoint is healthy, and jev_guard_status to see how the auto-guard is behaving.

Direct tool call shape

{
  "state": "I was double-charged for my subscription and I want a refund.",
  "questions": [
    { "name": "department", "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": { "billing": "Payment or subscription issues",
                    "technical": "Bugs or integration problems",
                    "sales": "Pricing or account questions" } },
    { "name": "is_urgent", "type": "noul",
      "instructions": "The message conveys urgency or time-sensitivity" }
  ]
}

Returns per-question answers (choice/score/noul + confidence + probabilities), usage, latencyMs and estimatedCostUs.

How the harness decides to call Jev

There are four ways a Jev call happens; 0.7.4 made the first one reliable, and 0.8.4 added the fourth:

System-prompt guidance (proactive). The plugin registers a prompt section named tool:jev (order 3000, override with guidance.order, disable with guidance.enabled: false, extend with guidance.extra). It states the per-tool trigger rules in priority order — if the decision falls into one of these classes, call this tool now — and is register-only-when-jev_decision-exists, so it never advertises tools that are not loaded. This is the mechanism that makes the agent reach for Jev by itself; without it a model only ever sees the tool list and rarely spends a call on it.
Tool descriptions (discovery). Each tool description carries the same wording: measured latency and cost, the question-type rules, "batch related judgments into ONE call", and an explicit boundary — jev_decision returns verdicts and never prose; jev_choose scores but does not explain.
Auto-triage (plugin-initiated, 0.8.4). With autoTriage.enabled the plugin itself judges the first step of each turn and injects the routing advice as a plugin-sourced message; the agent does not have to decide to call Jev first, it simply reads the advice. Disable with autoTriage.enabled: false, widen the input floor with autoTriage.minChars, cap the per-session call count with autoTriage.maxCallsPerSession.
Hooks and user turns. With autoGuard.enabled, every guarded shell-like call is audited before it runs (deterministic blacklist, then Jev) — no model decision involved. And any user can just ask: "judge this ticket with jev_decision", "rank these three approaches with jev_choose", "run jev_verify".

If the agent still ignores Jev, check in this order: guidance.enabled is not false; the prompt section logged guidance registered in the host output; jev_decision is in the tool list (jev_guard_status reports the tools it guards, not the registered set); and the task actually is a judgment call rather than a writing/reasoning task.

Auto-guard

When autoGuard.enabled: true, two hooks run next to every guarded tool call:

Safety (tools/pre-execute): a free deterministic layer runs first and splits its matches into hard (block outright) and soft (ask Jev). The seven hard rules cover catastrophic, irreversible operations: recursive deletion of a filesystem, drive or directory tree, disk formatting, database destruction, credential exfiltration, force-pushed git history. The three soft rules cover operations that are destructive but often legitimate: host restart / power-off, git-history rewrites short of a force push, and always-true DELETE/UPDATE conditions. Hard patterns only fire in executable position (start of the command, or right after ; / | / & / ( / newline, with only wrappers, env assignments and flags before them), so a command you merely quote or describe is never hard-blocked; a hard match found anywhere else is downgraded to a Jev-judged hint. Everything not blocked goes to Jev (risk noul ≥ denyThreshold ⇒ deny) with a bounded verdict cache (200 entries) and a per-session budget; Jev failure is fail-open with a warning.
Loop (tools/post-execute): consecutive same-tool calls with long outputs trigger a Jev stall judgment; on a stalled verdict a non-blocking advisory is injected into the next request, then a cooldown applies.

Every verdict — deterministic or Jev — is recorded in the session ledger and counted by jev_guard_status, which also surfaces any registration failure instead of letting it scroll past. Since 0.8.4 the Jev verdicts of the guard are metered too (onJevCall → kind: "judge"), so a busy guard shows up in jev_usage instead of leaving the panel at zero. Evidence from the 0.7.5 audit itself (2026-10-04): three review payloads were denied by the 0.7.4 rules — two by the deterministic layer, one by Jev at 81–85% (threshold 0.8) — which is exactly why the hard rules became position-aware. The 0.7.5 tests re-verify the 0.7.4 contract: the shell and PowerShell recursive-delete cases, the force-push case and the credential-read case still block deterministically (27/27 tests green).

Watching it work (no extra page)

Visualization lives inside the conversation — no separate tab:

Each tool ships a presentationMeta projection, so results render as structured cards rather than raw JSON:

jev_overview — a decision board: status chips (model / key / guard / threshold), eight stats (decisions, verifies, denials, advisories, median latency, average confidence, input tokens, total cost) plus the latest decisions and guard events.
jev_verify — the live accuracy / latency / calibration report: accuracy, correct count, high-confidence accuracy, median & p95 latency, tokens, cost and the mislabelled cases.
jev_guard_status — guard cards: guarded tools, deny threshold, session budget, safety & loop counters and the last guarded tool.
jev_usage — the usage/quota panel: quota status, today's calls and cost, three budget meters, a projected daily burn rate, a per-day sparkline over the retained history and the per-tool breakdown, with the honest boundary printed at the bottom.

The session ledger behind jev_overview always records (in memory, bounded at 300 events) — since 0.7.5 it no longer waits for the standalone page. That page still exists for deployments that want it: set dashboard.enabled: true and open http://127.0.0.1:3080/jev; only then are events also appended to $DSH_HOME/jev-roll.jsonl for external tooling.

The same ledger feeds the usage/quota panel (jev_usage, the usage block in jev_overview, and the board's quota section). It measures this machine only: TypeSafe exposes no balance endpoint, so the panel never claims a provider-side balance — only measured calls, tokens and cost against the local budgets you configure.

Verification (measured, dated)

See docs/verification.md for methodology and the latest dated results.

Status: ✅ verified against the live API on 2026-09-21 (jev-latest): accuracy 96.3% (26/27) with the guard-inclusive benchmark, median latency 283–308 ms across runs, ≈ $0.0004 per 27-question run. The only mislabel is the documented boundary near-miss (severity score 0.01 vs expected 0).

Latest re-measurement (2026-10-04, 0.7.5): 96.3% (26/27) on jev-latest (jev-1.13.0), median 266–440 ms, p95 825–1468 ms, 8,696 input tokens ≈ $0.000365, and 4.0–5.0 s wall for all 27 calls at concurrency 6 (3.999 s on the 2026-10-04 re-run). The single mislabel is the recorded boundary case (severity-low: 0.01 vs expected 0).

Reproduce any time:

TYPESAFE_API_KEY=... node bench/bench.mjs            # 1 pass (27 questions)
TYPESAFE_API_KEY=... node bench/bench.mjs --repeat 3 # latency stability

or ask the agent: “run jev_verify”.

0.7.4: the guidance section registers for real (root cause: a non-existent TOOL_JEV order slot made systemPrompt.section() throw, and the throw was swallowed) — see docs/verification.md. npm test covers the registration path.

0.7.5: the audit batch (server, client, packaging) — see the 0.7.5 section above and docs/verification.md; npm test is 27/27.

0.8.0: the usage/quota panel — locally measured usage with rolling windows, projections and optional hard budgets; npm test is 28/28. See the 0.8.0 section above. docs/verification.md records the measured accounting.

0.8.1: the registered settings schema is plain (no volatile marker), so the Settings > Plugins card decodes again and never reports a configured key as unconfigured; the card also falls back to the raw config layers when a value cannot be decoded. npm test is 29/29. See the 0.8.1 section above.

0.8.2: /jev stopped 404ing (normalized dashboard.basePath, a status page even when the board is off), a read-only /jev/api/usage route always mounts, and the composer gained a compact Jev · 今日 N 次 · $… usage pill with an expanding popover. npm test is 29/29. See the 0.8.2 section above.

0.8.3: settings saves actually reach the schema — nested path segments committed in one atomic mutate, plus a one-time repair of documents written by <= 0.8.2 with literal dotted keys. Client-only: refresh the page. npm test is 29/29. See the 0.8.3 section above.

Regression (2026-09-28, v0.7.0): in a headless profile with autoGuard.enabled: true, jev_guard_status reports the armed guard (tools list, deny threshold 0.8, budget) and jev_overview returns its board without the former credentialRef crash — the volatile-config unwrapping fix. npm test passed 19/19 at that time (29/29 today).

Configuration

Key	Default	Meaning
`apiKeyEnv`	`TYPESAFE_API_KEY`	credential-ref / env var for the key
`apiKey`	—	literal key override (secret)
`baseURL`	`https://api.typesafe.ai/v1`	API base
`model`	`jev-latest`	model (pin e.g. `jev-1.13.0`)
`timeoutMs`	15000	per-call timeout
`maxQuestionsPerCall`	25	question cap per call
`verifyEnabled`	true	register `jev_verify`
`guidance.enabled`	true	inject the "when to call Jev" section into the system prompt
`guidance.order`	3000	order of that prompt section (must resolve to a finite number)
`guidance.extra`	`""`	deployment-specific rules appended verbatim to the guidance
`autoGuard.enabled`	false	master switch for auto-guard hooks
`autoGuard.safetyCheck`	true	pre-execute risk check for guarded tools
`autoGuard.loopCheck`	true	semantic stall detection
`autoGuard.tools`	bash/pwsh/run_code/terminal	tools the safety check applies to
`autoGuard.denyThreshold`	0.85	Jev noul ≥ threshold ⇒ deny / advise
`autoGuard.maxJevCallsPerSession`	50	guard Jev budget per session
`autoGuard.loopConsecutive`	3	same-tool calls before loop check
`autoGuard.loopCooldownMs`	60000	cooldown after a stall verdict
`autoGuard.loopMinChars`	200	min result length for loop check

Related projects

dsh-jev — Jev guardrail suite (loop guard, safety gate, tool pruning) by zhangxaochen.
dsh-jev-tools — automatic Jev judgments + ledger by horusj.
browser-use/jev-ultrafast — browser agent on Jev.
APUS fast-browser-use — local re-implementation of the decision paradigm for browser automation.

This plugin focuses on what the others do not: clean full primitives, an auditable auto-guard, and an honest reproducible verification harness for the real TypeSafe API.

License

MIT

版本

0.8.6stable

2026/10/7

0.8.5stable

2026/10/7

0.8.4stable

2026/10/7

查看其余 13 个版本收起版本

0.8.3stable

2026/10/7

0.8.2stable

2026/10/6

0.8.1stable

2026/10/6

0.8.0stable

2026/10/6

0.7.5stable

2026/10/4

0.7.4stable

2026/9/30

0.7.3stable

2026/9/30

0.7.2stable

2026/9/28

0.7.1stable

2026/9/28

0.7.0stable

2026/9/28

0.6.0stable

2026/9/26

0.5.0stable

2026/9/24

0.2.0stable

2026/9/21

README

dsh-jev-verify

Jev — TypeSafe AI's System One decision model — as a first-class plugin for DeepSeek Harness (dsh).

Tool	What it does	Call it when
`jev_decision`	Up to 25 typed questions (`choice` / `score` / `noul`) about one `state`, answered in parallel in one call (published ~70–500 ms; measured median 266–484 ms and p95 825–1468 ms across our runs, with single-run maxima of ~1.5–5 s depending on network load). Every answer carries a calibrated confidence and probabilities; the result also reports model, latency, token usage and estimated cost.	You need fast, repeatable verdicts instead of prose: classification/labeling, routing or triage, priority/severity/satisfaction scoring, spam/toxicity/PII checks, intent or truth checks, extracting structured tags. Batch related judgments into one call — parallel, no extra latency.
`jev_choose`	Ranks 2–10 candidate options/approaches: each gets a fit score 0–3 and a risk noul, combined into a composite `fit/3 × (1−risk)`; returns an ordered table, a recommended pick, and per-option latency/cost.	Several independent approaches are on the table at a fork and you want a calibrated tiebreaker before deciding. Jev scores, it never explains — the reasoning stays yours.
`jev_batch`	Runs one set of 1–5 judgments over 1–20 texts: one real call per text (4-way concurrency by default, 6 max), returning a paste-ready markdown table sortable by a judged dimension; built-in `sentiment` / `priority` / `intent` / `pii` / `spam` presets, or your own questions in the `jev_decision` shape.	A batch of tickets/comments/logs/feedback needs labeling, classification, prioritization or PII/spam screening, or several candidates need ranking on one dimension — the tool for "many texts, same judgments"; never hand-roll one `jev_decision` per item. Each text is its own real call, and a failed row says 失败 with the raw error instead of guessing.
`jev_verify`	Runs the frozen 27-question labeled benchmark (urgency, spam, toxicity, personal data, routing, intent, search type, priority, severity, satisfaction, guard verdicts) against the live API at concurrency 6 (measured 4.96 s wall for all 27 calls in 0.7.5 and 3.999 s on a re-run, versus 10.1 s serial): accuracy overall and on the high-confidence subset, median/p95/min/max latency, calibration, tokens, cost, and every mislabeled case (total plus the high-confidence ones).	Endpoint health check, model-version comparison, or a regression check — never routinely: one run is 27 real API calls (~8.7K input tokens, ≈$0.0004).
`jev_guard_status`	Read-only audit of the auto-guard: deterministic rules and the Jev backstop counted separately (`checks` / `jevCalls` / `denied` / `deterministicDenied` / `auditCalls`), guarded tool names, `denyThreshold`, loop-check counts, remaining session budget.	Confirm the guard is armed, see how often it actually fired, or explain why a command was blocked. When the guard is off it says so instead of reporting zeros.
`jev_overview`	Read-only snapshot of this session's Jev ledger: recent decisions and choices with confidence, latency median & p95, question-type mix, guard events, cumulative tokens and cost, plus key/guard/threshold status.	The user asks what Jev has done, blocked or spent — or you need endpoint and budget state without restarting the session. The ledger starts at plugin-instance lifetime; guard counts live in `jev_guard_status`.
`jev_usage`	Locally measured Jev usage and (optional) local budgets: rolling windows (today / 7 d / 30 d / all / session), calls, input/output tokens, estimated cost, latency median & p95, question-type and per-tool mix, a projected daily burn rate, and — with `quota.enabled` + `quota.enforce` — a hard stop before the API call.	The user asks how much Jev has been used or what it cost, or you want to check a budget before a batch of judgments. Reads only local state — no API call. TypeSafe has no balance endpoint, so this is measured local usage, not provider credit.

Honest by design — no mock mode, no silent fallback:

without a TYPESAFE_API_KEY, tools and guard fail with explicit setup instructions (the guard fails open with a warning — it never silently pretends to have checked);
every jev_decision result includes the model, latency and token usage, so each call is auditable;
jev_verify refuses to report numbers it did not measure;
the benchmark CLI (bench/bench.mjs) is dependency-free and reproducible with any key.

What's new in 0.8.6

Fixed a dependency mistake that could stop the host from starting (issue #1): @deepseek-ai/dsh-tools was declared as a normal dependency. In a profile using nodeLinker: hoisted + hoistPattern: ["*"], pnpm materialised a real copy under the profile's node_modules/@deepseek-ai/; the tools loader entry resolved to that copy instead of the host's own, it failed to import its peers, and startup ended in 1 required plugin did not activate. It is now an optional peer (^0.1.5-rc.2) — never installed, so it can never shadow the host. dependencies now carries only the leaf library @deepseek-ai/schemastery.
New guard test test/manifest.test.mjs fails if any host runtime package reappears in dependencies (verified by reintroducing the bug and watching it fail). Suite: 38/38.
Otherwise identical to 0.8.5.

What's new in 0.8.5

"Remaining" finally has an honest source — the balance you declare, minus locally measured spend — instead of a number something made up.

A restart no longer zeroes the total. quota.persist now defaults to true: the ledger is written to $DSH_HOME/jev-usage.json, falling back to ~/.dsh/jev-usage.json when the host leaves DSH_HOME empty (that missing fallback was the root cause of the cumulative total always reading 0).
Remaining = declared − measured. TypeSafe exposes no balance endpoint at all (/v1/usage, /v1/quota, /v1/account, /v1/balance, /v1/credits, /v1/billing, /v1/me, /v1/limits, /v1/wallet, even /v1/openapi.json all 404, and /v1/models carries no quota/credit headers), so the provider side cannot be read. Set quota.declaredBalanceUsd and quota.balanceSince (e.g. 2026-10-01) and the panel prints 剩余 = 自报 $5.00 - 本机实测 $0.000004（2026-10-01 起）; leave them empty and it says the balance was not declared rather than guessing one.
The pill re-reads every 20 s (was 60 s), so a call you just made shows up while you are still looking at it.
The collapsed pill leads with 剩余 once a balance is declared (Jev · 剩余 $4.9997 · 今日 3 次), and the popover renames the cumulative row when the ledger is on disk.
New jev_batch — one call, one set of judgments, over a batch of texts, returned as a table. 1–20 items × 1–5 questions, one real Jev call per item (4-way concurrency by default, 6 max), returning a paste-ready markdown table; sort=auto|desc|asc|none orders rows by a judged dimension (auto only when the single question is a score, descending). Built-in sentiment / priority / intent / pii / spam presets, or your own questions in the jev_decision shape. A failed row is marked 失败 with the raw error — never guessed — and every row is metered separately (kind: "batch"), so N items really are N calls and N costs.
The settings card gained the two declaration fields; npm test is 37/37 (the new test/batch.test.mjs adds 7 cases: one real call per item with measured cost, the three sort modes, an unguessed failed row, a whole-batch failure that throws, preset expansion with the shared context prefix, 11 rejected argument shapes that spend no call, and the concurrency cap).

What's new in 0.8.4

The panel stops implying a quota that does not exist, the guard's own Jev calls finally reach the ledger, and the plugin now participates on its own.

No fabricated quota. TypeSafe is a metered API with no balance endpoint. With no local budget configured, the pill and the status page no longer print a "额度正常" verdict, no longer draw percentage bars against "∞", and no longer show "额度重置 13 小时 41 分后" — a countdown to a budget that was never set. Without a budget the panel states one thing: pay-as-you-go, nothing resets, only locally measured usage is reported. A "本地预算重置" countdown appears only when a local limit really exists, and it is labelled local.
Money first. The collapsed pill reads Jev · 今日 $0.000534 · 29 次, and the popover leads with today's cost and cumulative cost; call counts are secondary rows. Cost is what a pay-as-you-go user actually pays.
The guard's Jev calls are metered. judgeRisky/judgeStall used to answer the guard and vanish: a session could report safety.jevCalls: 2 in jev_guard_status while the usage ledger still read "今日调用 0 次". Both verdicts now report latency and token usage through an onJevCall event and are recorded as kind: "judge" ledger entries (tool: "auto-guard").
Auto-triage (autoTriage, on by default). On the first step of every turn one small Jev call (4 routing questions, ~1.5K input tokens, bounded by autoTriage.timeoutMs) classifies the request — intent, whether an atomic judgment is really needed, which kind, and which Jev tool serves it — and the advice is injected back as a plugin-sourced user message. It runs at most once per (agent, turn), never blocks a step, and fails open on an unknown host shape, a missing key, a timeout or any API error. Being metered, it is visible in the panel, so a working plugin no longer shows zero calls.
Harder guidance. The system-prompt section now carries mandatory-trigger rules (classification/routing, priority/severity/satisfaction scoring, truth and compliance checks, structured extraction, choosing among 2-10 options), forbids bypassing Jev "because I already know the answer", and states the real cost magnitude (~$0.00002-0.0001 per call; 25 questions cost the same as 1).
Tests: new test/triage.test.mjs plus extended assertions in test/usage.test.mjs, test/client.test.mjs and test/guard.test.mjs; npm test is 30/30. The client half takes effect on a page refresh; the server half (guard metering, auto-triage, guidance, status page) needs a host restart.

What's new in 0.8.3

Settings you save now actually take effect.

Root cause. The card handed dotted field names to scope.set(field), and that call stores path: [field] — one segment. A field such as dashboard.basePath was therefore written into the settings document as a single literal key named dashboard.basePath, which no schema ever resolves. Saving reported success and changed nothing: on a live host ~/.dsh/settings.yaml held the flat keys autoGuard.maxJevCallsPerSession: 60 and dashboard.basePath: jev while the resolved document still showed the schema defaults (50 and /jev). Found by reading the live document over the settings/describe RPC after the 0.8.2 restart.
Fix. The card builds { op: "set", path: [<segment>, …] } operations and commits them in one atomic scope.mutate(ops) call; it only falls back to chained scope.set on hosts that expose no mutate.
Repair on open. A one-time mount effect rewrites documents written by <= 0.8.2 that contain literal dotted keys: for each one, set the nested path and unset the literal key, in one atomic mutation, guarded by a WeakSet so it runs once per scope and wrapped so a failed repair can never break the card.
Client-only: refresh the page — no host restart. test/client.test.mjs PASS 7 covers path splitting, the generated operations, the atomic repair and its idempotence. npm test is 29/29.

What's new in 0.8.2

Two things the board got wrong, plus a usage surface that does not need the board at all.

/jev never 404s again. dashboard.basePath used to be taken verbatim, so a value such as jev (no leading slash) registered the literal path jev, which no browser request ever matches. Paths are now normalized (leading slash added, trailing slashes trimmed, empty or / falls back to /jev) and both /jev and /jev/ are registered. With dashboard.enabled: false the path still answers with a live status page — today's calls, cost, this instance, today's tokens and latency, quota state — that points at the composer pill and says how to turn the full board on, instead of a bare host 404.
/jev/api/usage — a small read-only JSON route that always mounts (it does not wait for dashboard.enabled) and projects the same locally measured snapshot the jev_usage tool returns.
A composer usage pill (conversation.input.right, beside the submit button), modeled on dsh-opencode-go: Jev · 今日 29 次 · $0.000534 with a warn/danger tone dot, expanding into a 270 px popover — today's calls and cost against their limits, this instance's calls, today's tokens, median and p95 latency, guard denials, the top three tools, time to reset, last refresh and a manual refresh. It polls every 60 s and on tab focus, falls back to the full board API when the pill route is absent, and degrades to Jev · 额度不可用 carrying the exact error — it never throws. It reports local measurements only, never a vendor balance; jev_usage in the conversation stays the full panel.
Regression tests: test/dashboard.test.mjs covers the path normalizer, the route table with the board on, off and on a custom basePath, the 200 status page and a failing snapshot; test/client.test.mjs covers the pill payload's three shapes, the label and tone states, and the rendered popover text. npm test is 29/29.

What's new in 0.8.1

Fixes the Settings > Plugins card that showed 未配置 / unconfigured and blank fields while a key was configured.

One field definition, two schemas. buildConfig() builds both Config (entry form; keeps the marker for hosts that understand it) and SettingsConfig (plain — the one registered with the settings service), so the 23 fields cannot drift apart.
The card no longer trusts a single decode path. When the decoded value is empty it reads the raw base/user layers the service already sends, says so on the card ("服务端配置值未能解码…"), and stays editable and savable.
A credential reference is not "configured". The badge has three honest states: literal key / 环境变量 <NAME> (unverified) / none.
Regression tests: test/config.test.mjs asserts the registered schema is plain and JSON-safe on the wire; test/client.test.mjs renders the card against an undecodable snapshot and asserts the badge names the configured reference and never says 未配置. npm test is 29/29.

Reproduced on the wire (deployed plugin, dsh 0.1.5-rc.2, 2026-10-06):

schema	empty-object fields on the wire	GUI decode
`Config` (entry-form only)	8 + every `quota.*`	FAILS — `$.enabled expected boolean but got [object Object]`
`SettingsConfig` (registered)	none	passes; `apiKeyEnv="TYPESAFE_API_KEY"`, `enabled=true`, `autoGuard.denyThreshold=0.8`

What's new in 0.8.0

The usage/quota panel: Jev usage is now measured, stored and surfaced — as a tool view, in the overview card, and on the standalone board.

jev_usage — a usage/quota panel you can call. Every decision, choice, verification and guard event is recorded locally with its measured input tokens, cost, latency, question-type mix and tool name. The tool returns rolling windows (today / 7 days / 30 days / all / session), a daily call and cost series, a projected daily burn rate ("at this pace the daily limit is reached in N h"), and an optional hard stop.
Local budgets, not invented provider balances. TypeSafe publishes no balance or quota endpoint (GET /v1/usage, /v1/quota, /v1/account, /v1/balance, /v1/credits, /v1/limits, /v1/billing … all 404; only /v1/models answers). So the panel reports only what this machine actually measured, against limits you set yourself: quota.dailyCallLimit, quota.dailyCostLimitUsd, quota.sessionCallLimit, plus warnAtPercent. The boundary is printed in both the text result and the card — never a fabricated balance.
Optional hard stop (quota.enforce). With enabled + enforce, a call that would cross a configured limit is refused before the API request, with a message naming the budget. Display-only by default.
Persistence (on by default since 0.8.5; opt-in in 0.8.0–0.8.4). 0.8.0–0.8.4 kept everything in memory (400 samples) unless you set quota.persist: true; since 0.8.5 up to quota.historyDays (default 30) of daily history is written to $DSH_HOME/jev-usage.json by default (atomic tmp+rename, fail-open), so the cumulative total survives a restart.
Surfaced everywhere. jev_overview gained a usage block, the standalone board (dashboard.enabled) gained a quota section with bars, projections and a 30-day sparkline, the settings card gained a "usage/quota" group, and jev_verify now records its 27 calls, tokens and cost instead of only the accuracy string.
Tests: 28 passing — the new test/usage.test.mjs covers the empty state, token accounting, batch calls, quota status and the hard stop, persistence across a day boundary, reload, reset and every formatter.

What's new in 0.7.5

Server

The session ledger records even when the dashboard is off. record() used to return early unless the standalone page was mounted, so jev_overview reported 0 calls / 0 tokens / $0 forever while jev_guard_status counted the same events. The in-memory ring now always records; only the HTTP page and the $DSH_HOME/jev-roll.jsonl append stay behind dashboard.enabled (this README and the setting text now say exactly that).
The guard reports the events it already knew about. The Jev denial path now emits onSafetyDeny (with confidence and the tool name) and the loop path emits onLoopAdvisory, so the dashboard advisory counter and the Jev-verdict branch of the ledger are no longer dead code.
Auto-guard rebuilt in two deterministic tiers with position awareness (see Auto-guard below): scanning a whole argument list with regular expressions is gone; the root-wipe finder walks flag tokens one by one, so extra flags, --flag=value forms and reversed flag order can no longer slip a root wipe past; a recursive delete of an ordinary relative directory (for example a build output folder) is a Jev-judged soft hint instead of a hard block; and a hard pattern that only appears in quoted prose is downgraded the same way.
TYPESAFE_BASE_URL / TYPESAFE_MODEL overrides work. The schema defaults were being read as if they were user settings, so the environment variables never won and self-hosted endpoints were silently sent to the official base URL.
jev_verify runs in batches of 6 (VERIFY_CONCURRENCY): measured 4.96 s (3.999 s on a re-run) wall for the same 27 calls that took 10.1 s serially, and it no longer flirts with the tool own timeout.
mislabeled is complete again and a new mislabeledHighConfidence lists which misses were high-confidence (previously one high-confidence miss hid every low-confidence one).
Errors name the tool that failed. missingKeyError() and the abort / timeout / HTTP messages were hardcoded to jev_decision even when jev_choose or the playground raised them.
Registration failures surface instead of scrolling past. safe() now accumulates regFailures, and jev_guard_status returns them as registrations.failures / failureCount — the failure class that shipped two broken releases.
(), and the Jev verdict cache is bounded at 200 entries.

Client (in-chat views and the settings card)

noul confidence is no longer inverted. A "no" verdict with noul: 0.02 used to display as "no · confidence 2%" in an alarm colour; it now shows the decision-side confidence (98%), matching the overview board.
The overview card shows average latency and the question-type mix that the presentation projection had been sending all along.
jev_choose is auditable: the tool view keeps the ranked options and the submitted candidate text plus the selection context, instead of only a count.
The running chip is honest: only jev_decision / jev_choose claim "0 questions" / "0 candidates" while a call is in flight; other tools show none.
Dead CSS variables fixed (--dsw-alias-bg-l2 → --dsw-alias-bg-layer-2): the recommendation row and chips were invisible in the dark theme.
The dashboard page reports its own failures instead of swallowing them into a permanent "loading…", and tolerates events without a timestamp.
The credential badge is three-state: a literal key shows "configured", an environment-variable reference shows "environment variable (not yet verified)", and neither shows "not configured" — an env name is no longer mistaken for a working credential.

Packaging and docs

test/functional.mjs → test/functional.test.mjs, so the real HTTP-contract and grading tests actually run under npm test (they never did: the glob only matched *.test.mjs).
bench/choose-e2e.mjs no longer hardcodes a developer home directory and reads the key from TYPESAFE_API_KEY or the DSH env file.
bench/bench.mjs takes --concurrency (default 6) and reports wall time; the user agent is versioned.
README: latency tail percentiles and the test count are stated as measured, engines.dsh is marked advisory, and the file no longer advertises a blacklist that scanned text it had never inspected.
Optional peer dependencies declared (dsh-credentials, dsh-client-locale, dsh-client-ui-settings, dsh-api-remotes).

What's new in 0.7.4

jev_choose finally registers — in 0.7.3 the tool did not exist in any host. Its options array parameter declared minItems/maxItems, keywords the host's value-schema DSL rejects (JsonSchemaError: unsupported JSON schema: parameters.options.minItems is not supported by the value schema DSL). defineTool threw, the per-tool quarantine swallowed the throw, and the release shipped a headline feature that no host could call — the boot test only asserted four tools, so nothing failed. 0.7.4 drops the unsupported keywords (the 2–10 option count, the ≤800-character options and the ≤2000-character context stay enforced at runtime by validateOptions in lib/counsel.js) and test/boot.test.mjs now asserts all five tools register through the real defineTool schema compiler.
The guidance section now actually registers — this is what makes the harness call Jev on its own. Until 0.7.3 the plugin asked the host for a section order via getSectionOrder("TOOL_JEV"), a slot that does not exist in the current DSH generation; the lookup returned undefined and the host's systemPrompt.section() rejects a non-finite order (TypeError: prompt section "<name>" order must be a finite number). The throw was swallowed by the same quarantine that protects the profile, so the prompt section silently never existed and the model only ever saw the tool descriptions. 0.7.4 resolves the order as getSectionOrder("TOOL_JEV") → guidance.order → 3000, logs the outcome (system prompt: guidance registered | section tool:jev | order N | chars M), and test/boot.test.mjs now fails if the registered order is not a finite number.
Sharper functional descriptions (what / when / when not). All five tool descriptions now state the measured latency, the question-type rules, the batch-in-one-call advice, the cost of jev_verify, and an explicit do not use for boundary (jev_decision is verdicts, never prose; jev_choose scores without explaining). The system-prompt guidance is generated from the same wording, so the prompt and the tool list cannot drift apart.
New guidance config group: guidance.enabled (default true), guidance.order (default 3000), guidance.extra (deployment-specific rules appended verbatim) — editable in the GUI settings card.

What's new in 0.7.3

jev_choose — multi-option counsel. New tool that ranks 2–10 candidate approaches in one call: each option is scored by Jev (fit score 0–3 with calibrated confidence + risk noul), combined into a composite (fit/3 × (1−risk)); returns an ordered table, a recommended pick and honest per-option latency/cost. Rendered in-conversation as a ranking table (ChooseBody); its guidance text was intended for the agent system prompt, but that registration silently failed until 0.7.4 (see below), and included in the dashboard/decision-board stats (kind: choose merged into decisions). Caveat: 0.7.3 shipped this tool broken — see the registration fix in 0.7.4 below.
Deployed 0.7.3 to D:\lab\jev (vendor) and the web profile pnpm store with byte-identical MD5 on all five changed files; node --check green; rankOptions unit-verified against a mock transport.

What's new in 0.7.2

Benchmark claims now match the code. jev_verify's tool description said "24 questions across 15+ cases" while lib/cases.js has held 27 labeled questions (27 cases, guard verdicts included) since v0.2.0. The description, README, market PR copy and the verification report now all state 27.
Re-measured on 2026-09-28 (run 4, before this release): 96.3% (26/27) — the same single recorded boundary miss as run 3 — 8,696 input tokens ≈ $0.000365, median 484 ms (network-dependent). Full terminal log: bench/run4.log.

What's new in 0.7.1

Docs sync (0.7.1 ships 0.7.0 code). The verification report now carries the complete measured history instead of only the latest run: the 2026-09-21 benchmark/guard runs, the 2026-09-23 GUI card, settings-card redesign and in-chat tool views, the 2026-09-26 two-generation settings-API compatibility work, and the 2026-09-28 v0.7.0 regression on a headless profile. docs/verification.md is the full 208-line record; no behavioural change versus 0.7.0.
Structured in-chat views for every tool. Tool results now carry a presentationMeta projection (kind: decision | overview | guard | verify), so the conversation renders a decision card, a guard board and a benchmark report — answers, confidence bars, status chips and stat grids — instead of raw text. The view falls back to parsing the tool text when meta is absent.
Full plugin settings card. Settings > Plugins > Plugin configuration > Jev now covers every option: credentials (key, credential-ref, base URL, model, timeout, question cap), tool toggles, the whole auto-guard block (safety/loop switches, guarded tool list, deny threshold, Jev budget, loop tuning) and the dashboard (enable + base path) and the usage/quota panel (enable, enforce, warn threshold, daily call / daily cost / session limits, persistence, history days) — with numeric validation and dirty-state handling.
Config handling hardened. Volatile config fields (schemastery .volatile()) are unwrapped before use, so an object-shaped apiKeyEnv no longer crashes credentialRef(...), autoGuard.enabled / dashboard.enabled actually take effect, and numeric options are read as numbers.
Tests: 28 passing (npm test) covering boot, client render, tool views, settings, dashboard, guard, usage/quota accounting and the presentation projections.

Why Jev

Install

dsh plugin --profile web add dsh-jev-verify

then add a loader entry to $DSH_HOME/profiles/<profile>/cordis.patch.yml:

- insert:
    - id: jev-verify
      name: dsh-jev-verify
      config:
        autoGuard:
          enabled: true     # opt-in auto-guard

Restart the profile (or use the plugin market's one-click install, which performs both steps).

Tip: for the dsh market UI install dshmarket first (dsh plugin --profile web add dshmarket) and install this plugin from Settings > 插件市场.

API key

Get a free key at https://console.typesafe.ai/keys. Then choose one:

export TYPESAFE_API_KEY=... in the launching environment, or
Settings > Plugins > Plugin configuration > Jev (credentials service), or
set apiKey in the plugin config.

Usage

Ask the agent to use jev_decision for any fast judgment. Example prompt:

Use jev_decision with state = "" and these questions: department (choice, criteria billing/technical/sales), is_urgent (noul), severity (score, 3 levels). Report answers + confidence.

Or request jev_verify at any time to confirm the endpoint is healthy, and jev_guard_status to see how the auto-guard is behaving.

Direct tool call shape

{
  "state": "I was double-charged for my subscription and I want a refund.",
  "questions": [
    { "name": "department", "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": { "billing": "Payment or subscription issues",
                    "technical": "Bugs or integration problems",
                    "sales": "Pricing or account questions" } },
    { "name": "is_urgent", "type": "noul",
      "instructions": "The message conveys urgency or time-sensitivity" }
  ]
}

Returns per-question answers (choice/score/noul + confidence + probabilities), usage, latencyMs and estimatedCostUs.

How the harness decides to call Jev

There are four ways a Jev call happens; 0.7.4 made the first one reliable, and 0.8.4 added the fourth:

System-prompt guidance (proactive). The plugin registers a prompt section named tool:jev (order 3000, override with guidance.order, disable with guidance.enabled: false, extend with guidance.extra). It states the per-tool trigger rules in priority order — if the decision falls into one of these classes, call this tool now — and is register-only-when-jev_decision-exists, so it never advertises tools that are not loaded. This is the mechanism that makes the agent reach for Jev by itself; without it a model only ever sees the tool list and rarely spends a call on it.
Tool descriptions (discovery). Each tool description carries the same wording: measured latency and cost, the question-type rules, "batch related judgments into ONE call", and an explicit boundary — jev_decision returns verdicts and never prose; jev_choose scores but does not explain.
Auto-triage (plugin-initiated, 0.8.4). With autoTriage.enabled the plugin itself judges the first step of each turn and injects the routing advice as a plugin-sourced message; the agent does not have to decide to call Jev first, it simply reads the advice. Disable with autoTriage.enabled: false, widen the input floor with autoTriage.minChars, cap the per-session call count with autoTriage.maxCallsPerSession.
Hooks and user turns. With autoGuard.enabled, every guarded shell-like call is audited before it runs (deterministic blacklist, then Jev) — no model decision involved. And any user can just ask: "judge this ticket with jev_decision", "rank these three approaches with jev_choose", "run jev_verify".

Auto-guard

When autoGuard.enabled: true, two hooks run next to every guarded tool call:

Safety (tools/pre-execute): a free deterministic layer runs first and splits its matches into hard (block outright) and soft (ask Jev). The seven hard rules cover catastrophic, irreversible operations: recursive deletion of a filesystem, drive or directory tree, disk formatting, database destruction, credential exfiltration, force-pushed git history. The three soft rules cover operations that are destructive but often legitimate: host restart / power-off, git-history rewrites short of a force push, and always-true DELETE/UPDATE conditions. Hard patterns only fire in executable position (start of the command, or right after ; / | / & / ( / newline, with only wrappers, env assignments and flags before them), so a command you merely quote or describe is never hard-blocked; a hard match found anywhere else is downgraded to a Jev-judged hint. Everything not blocked goes to Jev (risk noul ≥ denyThreshold ⇒ deny) with a bounded verdict cache (200 entries) and a per-session budget; Jev failure is fail-open with a warning.
Loop (tools/post-execute): consecutive same-tool calls with long outputs trigger a Jev stall judgment; on a stalled verdict a non-blocking advisory is injected into the next request, then a cooldown applies.

Watching it work (no extra page)

Visualization lives inside the conversation — no separate tab:

Each tool ships a presentationMeta projection, so results render as structured cards rather than raw JSON:

jev_overview — a decision board: status chips (model / key / guard / threshold), eight stats (decisions, verifies, denials, advisories, median latency, average confidence, input tokens, total cost) plus the latest decisions and guard events.
jev_verify — the live accuracy / latency / calibration report: accuracy, correct count, high-confidence accuracy, median & p95 latency, tokens, cost and the mislabelled cases.
jev_guard_status — guard cards: guarded tools, deny threshold, session budget, safety & loop counters and the last guarded tool.
jev_usage — the usage/quota panel: quota status, today's calls and cost, three budget meters, a projected daily burn rate, a per-day sparkline over the retained history and the per-tool breakdown, with the honest boundary printed at the bottom.

Verification (measured, dated)

See docs/verification.md for methodology and the latest dated results.

Reproduce any time:

TYPESAFE_API_KEY=... node bench/bench.mjs            # 1 pass (27 questions)
TYPESAFE_API_KEY=... node bench/bench.mjs --repeat 3 # latency stability

or ask the agent: “run jev_verify”.

0.7.5: the audit batch (server, client, packaging) — see the 0.7.5 section above and docs/verification.md; npm test is 27/27.

Configuration

Key	Default	Meaning
`apiKeyEnv`	`TYPESAFE_API_KEY`	credential-ref / env var for the key
`apiKey`	—	literal key override (secret)
`baseURL`	`https://api.typesafe.ai/v1`	API base
`model`	`jev-latest`	model (pin e.g. `jev-1.13.0`)
`timeoutMs`	15000	per-call timeout
`maxQuestionsPerCall`	25	question cap per call
`verifyEnabled`	true	register `jev_verify`
`guidance.enabled`	true	inject the "when to call Jev" section into the system prompt
`guidance.order`	3000	order of that prompt section (must resolve to a finite number)
`guidance.extra`	`""`	deployment-specific rules appended verbatim to the guidance
`autoGuard.enabled`	false	master switch for auto-guard hooks
`autoGuard.safetyCheck`	true	pre-execute risk check for guarded tools
`autoGuard.loopCheck`	true	semantic stall detection
`autoGuard.tools`	bash/pwsh/run_code/terminal	tools the safety check applies to
`autoGuard.denyThreshold`	0.85	Jev noul ≥ threshold ⇒ deny / advise
`autoGuard.maxJevCallsPerSession`	50	guard Jev budget per session
`autoGuard.loopConsecutive`	3	same-tool calls before loop check
`autoGuard.loopCooldownMs`	60000	cooldown after a stall verdict
`autoGuard.loopMinChars`	200	min result length for loop check

Related projects

dsh-jev — Jev guardrail suite (loop guard, safety gate, tool pruning) by zhangxaochen.
dsh-jev-tools — automatic Jev judgments + ledger by horusj.
browser-use/jev-ultrafast — browser agent on Jev.
APUS fast-browser-use — local re-implementation of the decision paradigm for browser automation.

This plugin focuses on what the others do not: clean full primitives, an auditable auto-guard, and an honest reproducible verification harness for the real TypeSafe API.

License

MIT