@playform/ets-hook-dsh-normalize-invisible
The DeepSeek Harness Plugin Family for PlayForm.
[!NOTE]
The invisible-character normalizer for model output - a DeepSeek Harness plugin that hooks the
llm/stream waterfall (the interceptable wrapper around EVERY streaming model call, bound to the
LlmRuntime) and normalizes the zero-width/invisible character family in model output, live in the
transcript - by REMOVING it: zero-width spaces, joiners, bidi controls and the BOM character
simply vanish (the default replacement is the empty string).
A CLASS flavor of the normalize family: the core's Invisible class plus a configurable
replacement string (default "" - removal).
The @-sentence identity: Hook @ DSH @ Normalize @
Invisible.
The family's raw-write tool (registered by
ets-hook-dsh-normalize-dash) bypasses this flavor's transforms too -
the exemption is family-wide: the core's Stream/Chunk + Stream/Block pass every raw-write
call through by identity, so the tool's explicit normalize parameter (default false = verbatim)
is the only normalization it applies.
Where It Fits
Family position (the @-sentence Hook @ DSH @ Normalize @ Invisible): a hook child of the
ets-plugin-dsh-factory service and the ets-hook-dsh-core
machinery; the fifth of the six stream normalizer siblings.
One of six siblings in the stream-normalization family, all built on the shared machinery -
ets-plugin-dsh-factory for State/Append (and the named Schema helper,
shared: false, for the config), ets-hook-dsh-core for the Invisible class and
the generic Replace/Chunk/Block dispatch:
A non-manifest factory consumer: it injects ["pluginFactory"] and uses State (cell
unwrap + shared Ledger/Enabled mappings + its own fields), Append and
Journal (the shared P5 record on every N > 0 pass); the config is composed
by the factory's standalone Schema helper with shared: false - the minimal block, no
fs/observed
dead fields.
It touches no files, so
fs/write-intent and
fs/observed never see it; it wraps the
downstream result and always calls next(), so it composes with other
llm/stream listeners
regardless of registration order.
The DSH plugin family is the DeepSeek Harness plugin layer of the PlayForm ecosystem:
TypeScript-first Source/ → Target/, the deterministic @playform build, prepublishOnly-only -
the same conventions as every other @playform package.
Install
Terminal (registry)
pnpm add @playform/ets-hook-dsh-normalize-invisible
- Remote / dependency install (auto-activation):
pnpm add @playform/ets-hook-dsh-normalize-invisible in a profile dir (+ the package name in
dsh.profile.bundles, or dsh plugin add) - installed automatically and activated at the next
host start.
- Local git clone: clone →
pnpm install inside the bundle (this bundle's own
pnpm-workspace.yaml - packages: [.], nodeLinker: hoisted - keeps the install local) →
npm run prepublishOnly to build Target/ → dsh plugin --profile <name> add <this directory>
and a restart.
Activation is at boot; verify with the ets-hook-dsh-normalize-invisible: activated (...) ledger
line (the --dump-config verification path is gone behind the app-managed profile guard - the
ledger and the runtime's inspect providers are the activation assessors).
Usage
The bundle is configured through its cordis.patch.yml row (or the profile's dsh.bundle
manifest) - the full config table is in The Config:
- insert:
- id: ets-hook-dsh-normalize-invisible
name: "@playform/ets-hook-dsh-normalize-invisible"
config:
log: true
logFile: ~/.dsh/ets-hook-dsh-normalize-invisible.log
replacement: ""
The shared Effect-TS plumbing - the runtime wiring, the stream machinery and the hook contract types - comes from the base ets-base-dsh package.
The Problem
Zero-width and format characters carry no visible width - which makes them the perfect smugglers.
A
zero-width joiner inside a file path, a right-to-left mark inside a command, the U+202E
visual-spoofing override in what looks like plain text: none of them show up in the transcript, all
of them change what a parser, shell or diff sees.
This flavor deletes the whole family before it
reaches the transcript.
How It Works
The stream pipeline
llm/stream waterfall (options, next) the interceptable wrapper around
│ EVERY streaming model call
▼ next() called FIRST, always - options never touched
Normalize(upstream, state) the async generator
│ for await (chunk of upstream)
▼
CoreChunk(chunk, transform, reasoning, toolArgs, raw) the core's per-chunk dispatch
│
├─ text-delta ────────► Replace(text, Invisible, "") rewrite the text field
├─ reasoning-delta ───► same, when normalizeReasoning (default ON)
├─ block-end ─────────► the assembled block's text fields -
│ TextBlock.text / ReasoningBlock.text (plus a
│ runtime `thinking` string field) - the deltas
│ AND the block must agree, or consumers see
│ inconsistencies
├─ tool-call-delta ───► Replace(argumentsDelta, Invisible, "") when
│ normalizeToolArguments (IMPLEMENTED, default
│ OFF - execution-critical raw JSON, the user's
│ accepted risk; the example patch turns it
│ on) - the assembled ToolCallBlock.arguments
│ follows the same flag via block-end
│ The gate is THREE-WAY with the flag on: a
│ delta/block whose `name` is "edit", "raw-write"
│ or "normalize-file" passes through BY IDENTITY
│ (the edit tool's `old_string` must match the real
│ file bytes; the raw-write tool owns its content
│ normalization; the normalize-file tool's arguments
│ carry a file path - a normalized dash inside a
│ filename would corrupt the target),
│ and a call whose arguments open with the
│ `{"__normalize":false` marker (FIRST key,
│ tracked per call id) passes through
│ UNNORMALIZED with the marker entry stripped,
│ so the executed call carries no unknown key
└─ block-start / usage / finish
──► PASSTHROUGH BY IDENTITY, ALWAYS
(usage/finish ordering is the
adapter contract)
│ count === 0 → original chunk BY IDENTITY; rewritten → shallow copy
▼
yield ──► downstream consumers = the live UI + the durable transcript
│ (order preserved, no buffering; upstream throws propagate)
▼ normal loop completion, Count > 0
Factory.Append ──► `ets-hook-dsh-normalize-invisible: normalized N invisible char(s) in one stream`
The transform
The core's Invisible class (see ets-hook-dsh-core), applied per text segment
through the core's generic class-to-string Replace:
[\u00AD\u200B\u200C\u200D\u200E\u200F\u202A-\u202E\u2060\uFEFF] → replacement
| U+ codepoints | Characters |
|---|
| 00AD | soft hyphen |
| 200B | zero-width space |
| 200C | zero-width non-joiner |
| 200D | zero-width joiner |
| 200E | left-to-right mark |
| 200F | right-to-left mark |
| 202A-202E | LRE, RLE, PDF, LRO, RLO (bidi embedding controls; U+202E is the classic visual-spoofing override) |
| 2060 | word joiner |
| FEFF | zero-width no-break space (the BOM character) |
All are removed (default replacement: "" - REMOVAL) per text segment:
- No context rules, chunk-boundary-safe - single-character replacement, no lookahead, per-chunk
application can never disagree with whole-text application.
- The replacement is applied with a function replacer, so a custom
replacement containing $
patterns is inserted literally.
- Replaced characters are counted per stream for the ledger line.
The Pitfalls
- Removal is the replacement -
replacement: "" (the default) removes the family (soft hyphen,
zero-width spaces and joiners, bidi controls, word joiner, BOM); a hot edit to a non-empty
replacement turns the pass into a replace instead.
- A dependency alone is NOT activation - the package must be in
dsh.profile.bundles (or the
patch row inserted), or no listener registers and no ledger exists.
- The stream gate rewrites model output only - it never touches disk; files already on disk are
the
normalize-file tool's job.
- The count is per replaced code point - N counts matched characters; N = 0 and an upstream
throw both write no count line and no journal record.
The Config
| Field | Type | Default | Volatile | Meaning |
|---|
log | boolean | true | yes | write the durable ledger file |
logFile | string | ~/.dsh/ets-hook-dsh-normalize-invisible.log | yes | the invisible ledger (separate from the family's logs) |
replacement | string | "" | yes | the transform's only knob - default REMOVAL; hot-editable |
normalizeReasoning | boolean | true | no | normalize reasoning deltas and the assembled reasoning block too |
normalizeToolArguments | boolean | false | no | IMPLEMENTED (default OFF): rewrite the tool-call argumentsDelta and the assembled ToolCallBlock.arguments when on (with three exemptions: edit, raw-write and normalize-file calls pass through by identity (the edit tool's old_string must match the real file bytes; the raw-write tool owns its content normalization through its explicit normalize parameter; the normalize-file tool's arguments carry a file path - a normalized dash inside a filename would corrupt the target); and a call whose arguments open with the {"__normalize":false first-key marker passes through unnormalized, the marker entry stripped) - execution-critical raw JSON, the user's accepted risk; the example patch turns it on |
Volatile cells commit without remounting the plugin; the factory's State builder unwraps them
defensively.
In Action
One stream, one deletion.
The model emits text carrying invisible smugglers; the live UI and the
transcript receive clean text - and the two lines below look identical in print, because the removed
characters are invisible.
The incoming line carries, in order: a soft hyphen (U+00AD), a zero-width
space (U+200B), a zero-width joiner (U+200D), a left-to-right mark (U+200E), a right-to-left mark
(U+200F), a word joiner (U+2060) and a BOM character (U+FEFF):
text-delta in (what the model wrote):
passsword: correct
text-delta out (what reaches the transcript):
"password: correct"
In the incoming line the smugglers sit inside the word password and around the colon; a diff would
see a password that does not match the author's text, a parser would tokenize differently, a shell
could splice a command.
After the pass all of them are simply gone - the output line is what the
reader should see.
When a stream finishes normally with replacements made, the ledger gets the count
line:
ets-hook-dsh-normalize-invisible: normalized 7 invisible char(s) in one stream
The replacement is hot-editable (the default "" removes; a non-empty value such as "?" would
mark each smuggler's position for debugging), and the same pass runs over reasoning deltas when
normalizeReasoning is on and over tool-call arguments when the example patch enables
normalizeToolArguments - with the edit, raw-write and normalize-file names exempt and raw-marker calls passing through
unnormalized.
The Ledger
Two lines, both written through the factory's Append (the ets-hook-dsh-normalize-invisible: prefix
is the logger's <State.Module>:; the durable file line is [<ISO>] <message>):
ets-hook-dsh-normalize-invisible: activated (replacement=, reasoning=on, toolArgs=off, logFile=~/.dsh/ets-hook-dsh-normalize-invisible.log)
ets-hook-dsh-normalize-invisible: normalized N invisible char(s) in one stream
The empty default replacement renders as the empty replacement=.
The activation line is written by
apply(); the count line only follows a normal stream completion and only when N > 0 (a thrown-away
stream writes no ledger line).
Every N > 0 pass also journals one normalized record into the shared package_governance v2
domain - event normalized, path = the empty stream-level path, detail byte-identical to the
count line - best-effort: with no storage facility the record buffers or drops, and the human
ledger stays the complete record.
License 📜
CC0-1.0.
