dsh-topics-memory
English | 中文
A dsh plugin: maintains "working topic memory" as an OKF (Open Knowledge Format v0.2) knowledge bundle, persisted in a local git repository (optionally synced to a private GitHub repo), with conclusions traceable through git history, sessions automatically observed and distilled into knowledge, and relevant topics injected to the model before every turn.
Requires dsh >= 0.1.5-rc.2 — this plugin targets the dsh RC/stable line only (CI and releases resolve the newest of the latest/next dist-tags at runtime). The alpha line is no longer supported.
Demo
The full flow in about four minutes: topics captured, distilled, and injected in a real session.
https://github.com/user-attachments/assets/8c06cc98-b1ed-402b-9110-4f9a93eb15bc
The problem it solves
Long sessions forget. Cross-session, even more so. This plugin maintains structured topic memory: each Topic records a matter's name, dependencies, open questions, current conclusion, impact, and recommendations. When a conclusion changes, edit the file and commit — git log directly answers "when, by whom, and why did this conclusion change".
Why this plugin exists: memory is edited, not accumulated
The short version: more memory is not better memory.
Most memory tools assume accumulation — record everything, retrieve broadly. That may work for humans; for LLMs it backfires twice over. Model attention is a finite resource, so a giant memory bank means every turn is spent digging for signal in noise. Worse, process memories hoard intermediate judgments that were right once and wrong later — and they will confidently steer the model into bad decisions.
So this plugin takes a hard editorial line on what deserves to be remembered: a topic records exactly four things — the question that started it, the conclusion it reached, what it impacts, and what it depends on. Everything in between — the discussion, the dead ends, the wrong turns — is deliberately not memory. Process belongs to the session; when the session ends, it goes. Only conclusions that survive distillation make it into the bundle.
Short-term memory is the session's own job — the conversation context already is one, and a plugin that feeds it back is noise. Long-term memory belongs to topics: small, structured, git-traceable, injected in budgeted slices with zero hits meaning zero injection. Every turn hands the model the minimum high-value context, not the biggest warehouse.
This plugin is not trying to be the model's notebook. It is trying to be the model's editor: deciding what is worth keeping — and, more importantly, what should be forgotten.
Core features
- Strict OKF v0.2 compliance: each Topic is a
markdown + YAML frontmatter concept document (type: Topic) that the whole OKF ecosystem (Obsidian, OKF validators) can consume directly; ships with the provenance (sources), trust (generated/verified), and lifecycle (status/stale_after) field families.
- Git-traceable: one conclusion change = one commit (write-through); the
topic_history tool and /topics history make change history first-class.
- Local-first: local-only mode by default (
~/.dsh/topics/), zero config, zero credentials; setting repo enables GitHub sync (single repo, single bundle, single main, write-through + debounced push; rebase conflicts are demoted and flagged for a human — no automatic smart-merge).
- LLM-free hot-path injection: per-turn lexical matching (CJK bigrams + words + weighted tags +
depends graph walk), millisecond-scale; zero matches = zero injection; per-topic digest ≤300 tokens, top-K ≤4, total budget ≤1.5k tokens — all configurable.
- Observable, tunable injection: every turn writes an Injection Log (hits, scores, near-misses, budget usage);
/topics stats reports hit rate, top-N, near-miss distribution, and tuning suggestions — tune from evidence, not vibes.
- Knowledge as a graph:
depends (machine-readable directed edges) plus body [[wikilinks]] and markdown links (human-written edges) form one graph; retrieval walks it in both directions (per-level decay, configurable depth) so a single hit pulls in a knowledge subgraph; every write rebuilds the meta/backlinks.json reverse index, and /topics show lists "who references me, and how" — check the blast radius before changing a conclusion.
- Two-stage observer (M2): the main model jots atomic observations with
topic_observe; a background distill lane (session end + every N turns, model configurable) distills them into formal Topics in batches; when the model itself deems something worth keeping, it topic_saves directly.
Quick start
- Install (command below), restart dsh;
- Run
/topics onboard — native dsh ask-user panels walk you through the five decisions: mode / repo / distill model / injection tier / auto-observe — nothing is written until the final confirm;
- Work as usual: relevant conclusions are injected every turn; say "remember…" to have the model
topic_save; /topics status for health, /topics stats for injection stats.
Tools & commands
| Model tools | Purpose |
|---|
topic_save | Save/revise a Topic (name / dependencies / open questions / conclusion / impact / recommendations) |
topic_observe | Jot an atomic observation (decision/finding/constraint/question), pending distill |
topic_search | LLM-free keyword search over memory |
topic_history | A topic's conclusion change history (git log as a tool) |
| Command | Purpose |
|---|
/topics onboard | Interactive setup wizard on dsh-native ask-user panels (mode / repo / distill model / injection tier / auto-observe); typed fallback where no ask-user UI exists |
/topics status | Bundle health: topic count, observation backlog, conflicts, last distill outcome, sync status |
/topics distill | Manually trigger one distill run over the current observation pool (same lane, same in-flight guard; summary mirrors the distill-state fields) |
/topics consolidate | Manually trigger one consolidation run: the LLM gardener merges duplicates, promotes settled drafts, deprecates superseded entries, refreshes metadata — every change is its own git commit, revert to roll back |
/topics stats | Injection stats: hit rate, top-N, near-miss distribution, tuning advice |
/topics list / show / history | Browse topics, backlinks, and change history |
/topics graph | Generate a relationship-graph web page (force-directed, draggable/zoomable, hover for conclusions) and open it in the browser |
/topics sync [pull|push] | GitHub mode: manual pull/push (automatic by default) |
/topics config / set <key> <value> | View and edit config (thresholds, budgets, distill model, …) |
Install
dsh plugin --profile <your profile> add @aiwayds/dsh-topics-memory
First thing after installing: run /topics onboard. The bundle lives at ~/.dsh/topics/ by default ($DSH_TOPICS_HOME overrides). GitHub sync: /topics set repo <owner/name> (suggested repo name dsh-topics-data, to keep it distinct from the plugin's own source repo); credentials come from $GITHUB_TOKEN or a logged-in gh CLI (login is not this plugin's job).
Upgrading from 0.5.x (rename)
0.6.0 renames the plugin: @aiwayds/dsh-llmwiki-memory → @aiwayds/dsh-topics-memory, the /wiki command family → /topics, and the settings namespace llmwiki → topics. Install the new package (and remove the old one from your profile) — on first start the plugin migrates everything automatically: the data directory ~/.dsh/llmwiki is renamed to ~/.dsh/topics, and user-tuned values in the old llmwiki settings namespace are copied into topics. No manual steps; if a migration step fails the plugin falls back to the old locations and keeps working.
Uninstall
Remove the plugin from a profile:
dsh plugin --profile <name> remove @aiwayds/dsh-topics-memory
The host reconciles the profile automatically: the dsh.profile.bundles entry is spliced and the patch layer is dropped.
What stays on disk (kept on purpose — this is your memory):
~/.dsh/topics/ — the whole topic bundle: topic markdown, meta/, and the embedded .git repo (the full history; it may carry an origin remote — GitHub sync stops with the plugin). To archive the bundle elsewhere, copy or clone this directory as-is.~/.dsh/settings.yaml topics: section — user overrides. Reinstalling silently reactivates sync including any configured repo; delete the section first if you want a clean start.- The legacy
llmwiki: settings section left by the 0.5.x → 0.6.x migration is never auto-deleted; remove it by hand once nothing needs it.
Purge everything: back up ~/.dsh/topics first, then rm -rf ~/.dsh/topics.
Configuration
First-time setup belongs to /topics onboard; day-to-day tuning is /topics set <key> <value> (writes the topics namespace in settings.yaml, effective from the next session). All keys and defaults:
| Key | Default | Meaning |
|---|
repo | empty (local-only) | GitHub sync repo owner/name; suggested dsh-topics-data; empty = back to local-only |
autoInject | true | Per-turn injection master switch |
injectDedup | true | Session-level injection dedup: topics already injected in this session are not re-injected (registry cleared at session end; budget-dropped topics stay injectable; deduped topK slots are NOT backfilled) — ADR 0012 |
suppressEcho | true | Distill-echo suppression: topics distilled from the CURRENT session's own turns are not injected back into it (provenance rides the observations log sessionId → distilledInto) |
topK | 4 | Max topics injected per turn |
perTopicBudget | 300 | Per-topic digest token budget |
totalBudget | 1500 | Total injection budget per turn |
matchThreshold | 0.3 | Hit threshold; tune from /topics stats near-miss evidence |
tagBoost | 0.15 | Additive boost per tag hit (total cap across hits equals this value) |
injectMode | pointer | Injection shape: pointer (light pointers, ≤80 tok each, topic_open pulls the full text; total budget capped at 600) / digest (full digest rendering, per-topic 300 / total 1500) |
qualityLane | sampled | Slow quality lane: off / sampled (1/3 of turns) / always; produced at turn/end, consumed by the next injection (consume-once), never for subagent sessions |
graphDepth | 2 | depends graph walk depth (0 disables) |
recencyWindowDays | 7 |
Acknowledgements
This project's shape is directly inspired and supported by:
- zosmaai/pi-llm-wiki — a native OKF v0.2 knowledge extension for pi and this project's direct inspiration; its two-stage observation (cheap atomic observations + background distill), cache-safe injection (volatile content never enters the system prompt), and layered vault & ownership model are all absorbed here.
- GoogleCloudPlatform/open-knowledge-format — the Open Knowledge Format (OKF) v0.2 spec this bundle format strictly follows.
- Karpathy's LLM Wiki pattern — the starting point of the whole "an LLM maintains a personal knowledge base" methodology.
- fan56/pi-topic-memory — the same author's predecessor: a working topic ledger with silent injection for pi; its LLM-free hot-path matching and injection-timing experience is this project's direct technical ancestor.
- chancelu/dsh-llmwiki — a fellow dsh-ecosystem precedent; this project's same-turn injection seam (
agent/inbox/spliced + systemPrompt.context()) follows the mechanism it validated on real dsh.
Known boundaries
- Subagents are out of memory by default — one switch to opt in: by default (
include-subagents off since 0.7.0) delegated sessions are skipped entirely — no injection, no observation, no distill triggers; /topics set include-subagents on applies injection and observation to them too. The topic tools stay on the global layer, so an explicit topic_save from a child still lands. Out-of-process subagents (claude-code/codex providers) never load this plugin anyway.
- Exit is local-only (0.10.0): the plugin's disposer makes one local git commit of the meta sidecars (observations / injections / distill state) and never waits on the network — no pull, no push, no model call, so host exit no longer pays a git round-trip or a bounded distill wait (the old 90s cap is gone; only a 10s guard against a pathological git stall remains). The exit distill trigger is fired fire-and-forget and is skipped entirely while a session-end run is still in flight (the same pool head would otherwise be fed to the model twice). Nothing is lost by skipping: observations are write-through on disk, the deferred push is replayed by the next boot's pull, and the skipped distill is replayed there too (boot-replay).
meta/distill-state.json records each lane's outcome, checkable via /topics status.
- Observation GC (three strikes): an observation the model actually evaluated (parseable answer, however useless) but no op consumed accrues one failed attempt; the third failed attempt physically deletes it — explicitly authorized cleanup of raw data the lane demonstrably cannot process. Runs that never evaluated the batch never count: infrastructure failures (network errors, unconfigured distill route → readable
no-model short-circuit) and unparseable output (invalid-output) are exempt, and a batch still mid-shrink on output-limit retries is only counted once a verdict is reached (success, floor stop, stall, or an explicit skip). Deletions are committed immediately (data destruction stays git-traceable); pure attempt counters follow the usual flush cadence.
- Config read timing:
/topics set and settings.yaml edits take effect most reliably from the next session start.
- Picking a distill model:
/topics onboard splits the distill decision into two dependent questions (provider first, then that provider's model catalog), pre-validated with resolveModelInfo — a provider with no live route blocks and re-asks, an off-catalog model (a non-NO_ADAPTER failure: outside the advisory catalog, possibly still usable) warns but is allowed; hosts without an ask UI or a usable model route fall back to typed input. The same validation backs the /topics set picker panels.
Design docs
- CONTEXT.md — domain glossary
- docs/adr/ — 0001–0013: OKF compliance, remote shape, sync strategy, two-stage observer, bundle layout, injection defaults, observability & tunables, dual-mode persistence, onboarding wizard, subagent isolation, the include-subagents switch, injection dedup default-on, the rename & migration
License
MIT
