dsh-context-diet
A global DeepSeek Harness plugin that removes tokens from tool output before
that output can enter a model request — without removing anything the model may
need, and without changing what the model is able to do.
tests/t0.py::case PASSED
tests/t1.py::case PASSED
…
[... 12235 of 12416 characters omitted by dsh-context-diet; 4 diagnostic lines
retained, middle elided ...]
______________________ test_charge_card_rejects_expired _______________________
src/payments/charge.py:118: in charge_card
E CardExpiredError: card expired 2024-01-31
FAILED tests/test_payments.py::test_charge_card_rejects_expired
[…]
…899 tests collected…
(Trimmed 12235 characters. Full normalized result stored at:
/…/session-…/bash.txt. Use read with offset/limit, or grep this path.)
The problem it solves
Every tool result becomes part of the conversation the moment it is logged, and
from then on the model re-reads it on every step of the session. The shipped
dsh-compaction-tool-result-pruner only trims once a compaction trigger
qualifies, and rewriting a result that late invalidates the prompt-cache prefix
that was already established — so trimming late costs extra as well as saving.
dsh-context-diet runs at tools/post-execute, the last waterfall before the
registry freezes a result into a tool/result event. A rewrite there belongs to
the call it came from: there is no cache prefix to invalidate, no second copy to
reconcile, and no durable edit to replay.
What it removes
1. Lossless normalization (on by default)
Reduces output to what a terminal would actually display. A terminal shows no
glyph for an escape sequence, so removing one cannot change meaning.
| Transform | Example | Result |
|---|
| Escape sequences | \e[31merror\e[0m | error |
| OSC hyperlinks and titles | \e]8;;http://x\a link | link |
| Carriage-return overlays | 10%\r50%\r100% | 100% |
| Backspace erasure | ab\bc | ac |
| Trailing whitespace | a \n | a\n |
| Blank-line padding | a\n\n\n\n\nb | a\n\nb |
2. Annotated compaction (on by default)
A run of at least repeatThreshold (8) byte-identical consecutive lines that is
also at least repeatMinRunChars (1000) characters long becomes one line naming
the exact repeat count:
npm warn deprecated inflight@1.0.6: This module is not supported… [x400 identical lines omitted by dsh-context-diet]
The run is collapsed only when the annotation is strictly shorter than the run
it replaces, so this pass can never enlarge a result. The count is preserved, so
the run is exactly reconstructible from the annotation and nothing is dropped
silently — which is why this pass needs no recovery file.
3. Recoverable, structure-aware trimming (on by default above 32,000 characters)
A result still larger than maxInlineChars is replaced by:
- its head (
headChars),
- an omission marker stating exactly how much was dropped,
- the diagnostically significant lines from the omitted middle — errors,
failures, exceptions, assertions, stack frames, diff hunks and headers, and
file:line:col references (significantChars), and
- its tail (
tailChars),
followed by a notice naming a spill file that holds the complete normalized text,
with the backend's own retrieval guidance.
Blind head/tail cutting discards the middle unconditionally. Keeping the
diagnostic lines makes the same budget strictly more informative, not less —
measured on a 900-test pytest -v run, every one of CardExpiredError,
charge.py:118, test_charge_card_rejects_expired, and 1 failed, 899 passed
survives with this plugin and is lost by a head/tail cut of the same size.
Why quality is not affected
- Nothing is dropped without a record. Every removal is either a
control character a terminal never displays, or something that is counted,
annotated, and written verbatim to a spill file the notice tells the model how
to read.
read is exempt by default. File-content output is line-addressed: blank-line
or trailing-whitespace rewriting would corrupt content a model may quote back
at an editor. This matches the shipped dsh-spill-policy, which exempts read
for the same reason.- The default budget is conservative. No result under 32,000 characters —
roughly 8,000 tokens — is touched at all, except by the lossless passes.
- It cannot alter capability. It registers no tools, removes none, changes no
prompts, and changes no model or reasoning settings. Tool schemas, the system
prompt, and the tool catalog are byte-identical to a run without it, so the
prompt cache behaves exactly as before.
- It never makes a model call and never adds a request.
- It fails open. Any unexpected shape, missing service, or thrown error
returns the incoming result untouched and logs a warning.
verify.mjs covers
each of those paths.
Measured effect
On a synthetic but realistic bash-heavy corpus (900-test pytest -v, npm install
with 400 repeated deprecation warnings, 2,400-line cargo build, a curl
progress bar, and a 3,000-entry find), with shipped defaults:
| Result | Before | Model sees | Removed |
|---|
pytest -v, buried failure | 48,002 | 12,555 | 74% |
npm install, 400 repeats | 33,711 | 236 | 99% |
cargo build, buried error | 158,997 | 20,218 | 87% |
curl progress bar | 4,572 | 61 | 99% |
find over a large tree | 130,890 | 12,218 | 91% |
| Total | 376,172 | 45,288 | 88% |
The lossless tier alone accounts for 15% of that; the rest is annotated
compaction and recoverable trimming. Character counts are not billed tokens —
the figures are indicative, and /diet reports the real numbers for the session
you are in.
Verified against a running Harness
Both passes were exercised live, through the real tools/post-execute waterfall
of a running dsh web profile, using the real dsh-spill-local backend:
$ for i in $(seq 1 30); do echo "SAME LINE PADDING PADDING PADDING PADDING PADDING"; done
SAME LINE PADDING PADDING PADDING PADDING PADDING [x30 identical lines omitted by dsh-context-diet]
$ awk '… 570 lines with an error at line 285 …'
download asset 0000 of 0570 ================================================
…
[... 31890 of 43967 characters omitted by dsh-context-diet; 1 diagnostic lines retained, middle elided ...]
src/engine/pipeline.rs:204:7 - error TS2345: UNIQUE_BURIED_DIAGNOSTIC_MARKER
[...]
…
download asset 0569 of 0570 ================================================
(Trimmed 31890 characters. Full normalized result stored at:
/…/c1cbcd405966-bash.txt. Use read with offset/limit, or grep this path to search within it.)
The 43,967-character result reached the model as 12,077 characters (72% smaller),
the diagnostic line that a head/tail cut would have dropped was retained, and the
recovery file was verified to hold all 571 lines of the original output.
Install
# 1. install the package into a profile
dsh plugin --profile web add link:/path/to/dsh-context-diet
- Enable the bundle: Settings → Plugins → Installed → dsh-context-diet → on.
An agent driving the plugin manager does both steps at once with
install_bundle.
The bundle inserts one global Host row, so it applies to every session in the
profile — not to a single session. It needs no credentials and no other plugin;
the tools/post-execute seam is part of dsh-tools, and the spill backend comes
from dsh-spill-local, both of which the shipped compositions provide. Without a
spill backend the plugin still normalizes and compacts, and simply declines to
trim, because trimming without a recovery file would lose information.
Configuration
Every field is optional; omitting it keeps the shipped default. Override any of
them in the profile's own cordis.patch.yml, which is applied after bundle
layers and therefore survives plugin upgrades:
- id: context-diet
name: 'dsh-context-diet'
config:
maxInlineChars: 64000 # 0 keeps only the lossless and annotated passes
exemptTools: ['read', 'read_image']
repeatThreshold: 12 # only collapse very long identical runs
| Field | Default | Meaning |
|---|
normalize | true | Master switch for the lossless passes |
stripAnsi | true | Remove terminal escape sequences |
resolveCarriageReturns | true | Reduce \r overlays to the final visible line |
trimTrailingWhitespace | true | Remove trailing spaces and tabs |
collapseBlankRuns | true | Cap runs of blank lines |
maxConsecutiveBlankLines | 1 | How many blank lines to keep |
collapseRepeatedLines | true | Annotate long runs of identical lines |
repeatThreshold | 8 | Minimum identical lines in a collapsible run |
repeatMinRunChars | 1000 | Minimum characters in a collapsible run |
maxInlineChars | 32000 | Trim above this size; 0 disables trimming |
headChars | 8000 | Leading characters retained by a trim |
tailChars | 4000 | Trailing characters retained by a trim |
significantChars | 8000 | Budget for retained diagnostic lines |
significantPattern | built in | Regex selecting the diagnostic lines to keep |
exemptTools | ['read'] | Tools the plugin never touches |
recoveryNoticePattern | built in | Recognizes a result that already names a recovery file |
maxInlineChars must leave room for the omission markers and the recovery
notice: headChars + tailChars + significantChars + 700 must not exceed it. A
config that violates this is rejected at activation with that arithmetic in the
message, rather than producing a trim that cannot converge.
Asking what it saved
Run /diet in a session:
dsh-context-diet — 138,204 characters removed (51.2%)
results inspected 93
characters in 270,163
characters out 131,959
removed, lossless 12,308
removed, annotated 4,120
removed, trimming 121,776 across 4 recoverable result(s)
estimated tokens saved 34,551 (chars / 4)
/diet reset clears the session's counters. The counters are plugin memory only:
nothing is written to the session log, so replay, fork, and resume are unaffected.
Interaction with the shipped plugins
dsh-spill-policy (shipped with maxInlineTokens: 12500) registers with
prepend: true, so it is the outermost listener and gets the last word. This
plugin runs after it and produces at most maxInlineChars characters — about
8,000 tokens, under that cap — so spill-policy finds nothing left to bound. The
recoveryNoticePattern check additionally makes this plugin stand down on any
result that already names a recovery file, so the two can be reconfigured
without producing two spill files for one result.dsh-compaction-tool-result-pruner still runs under compaction pressure.
It has strictly less to do here, which means character pressure is relieved
earlier and summarization is triggered later — the expensive part of a long
session.dsh-token-meter measures the request surface; this plugin makes that
surface smaller before the meter ever sees it.run_code / PTC sub-dispatch logs (tools/ptc-dispatch-log) are not
handled. PTC presentation is off in shipped compositions, and the extra seam
would double the maintenance surface for no benefit today.
How it stays working across Harness upgrades
The plugin is written to depend on as little as possible, and to degrade rather
than break:
- No Harness imports at all.
index.js imports only ./normalize.js.
There is no peer dependency on dsh-tools, dsh-llm, dsh-token-meter, or
@deepseek-ai/schemastery that a version bump could invalidate, and no build
step, so no compiled artifact can go stale.
- Only stable seams. The
tools/post-execute waterfall, the optional
spillStore service (contract: saveText(input) -> { locator, retrievalHint }),
the optional commands service, and ctx.logger. Each is a published Service
or Event contract, and each is used through ctx.get / ctx.inject so its
absence is a normal state rather than an activation failure.
- Config via the Standard Schema protocol.
Config implements
~standard.validate, which cordis validates natively, so no schema library
is loaded. Unknown or non-object config never throws inside apply.
- Fail-open everywhere. The handler awaits
next() first, so it never
swallows another plugin's decision, and it replaces only the rendered
projection — the registry merges it over result, so a tool's structured
value is untouched. A non-accept decision, an outcome with no text blocks,
an outcome whose shape is unexpected, or any thrown error returns exactly what
it received.
- No durable state and no new event types. The plugin writes no session
events, so a session it touched reopens, forks, and replays like any other.
Verifying it
node verify.mjs
52 checks cover the text transforms, the trim invariants, the config protocol,
and the plugin wiring — including the identity-preserving no-op path, the
recovery-file path, and every fail-open path (no spill backend, throwing spill
backend, malformed result, unknown session owner, missing command registry).
The script has no dependencies and does not need a Harness profile.
Editing the plugin while a Harness is running
A Harness process caches one module generation per plugin specifier for its whole
lifetime. Editing index.js, reinstalling, moving the package, disabling and
re-enabling the bundle, and even changing the package's exports map all leave
the already-loaded generation running; only a specifier the process has never
imported gets a fresh generation (install_bundle does load a brand-new package
live). So after changing this plugin:
- restart the Host (
dsh web) to load the new code, or
- install it under a new package name to load it immediately.
This is stock DeepSeek Harness behavior — "replacing an installed package
requires restart to load a fresh JavaScript module generation" — not something
specific to this plugin. If you install a new plugin and it appears inert,
check this before debugging the plugin.
