dsh-knowledge-sqlite
English | 中文
Chinese (README.zh.md) is the authoritative version; the English translation may lag behind.
Cross-session knowledge for DeepSeek Harness: a ctx.knowledge service seam plus knowledge_* model tools, backed by SQLite FTS5 trigram retrieval (CJK substring search) with L1 query expansion.
Implements the V1.11 knowledge seam proposal (11-round reviewed design, community proposal — not an official DSH feature). Contract: write/update/search/list/delete + budget + events + error codes.
dsh plugin add dsh-knowledge-sqlite
Why trigram FTS5
DSH's current full-text search uses unicode61, which cannot match Chinese substrings. Measured on the project's evaluation suite (see EXPERIMENTS):
| Retrieval stage | Hard rewritten queries (14, 512-item corpus) | Real human queries (17 targeted, 522-item corpus) |
|---|
| unicode61 (baseline, no CJK substring) | 0% | — |
| L0 pure lexical (trigram) | 7% | 65% |
| L1 (default): + query expansion | 21–64% (prototype live: 43–64%) | 65% (prototype live; reasoner variants measured 82%) |
| L2 (opt-in): + write-time enrichment | 50% | 88% (with n-enrichment) |
Acceptance gates (from V1.11): L1 recall@1 ≥ 30% on the hard set, ≥ 50% on the human set — both passed by the live prototype (kpro-2) and reproduced exactly by this bundle's SQLite layer (A 7% / C 21% / D 50% / human-A 65%, per-query rank identical).
What it does
knowledge_write — write a durable item; indexed immediately, zero LLM. dedupeKey upserts with stable id (hash(workspace+scope+dedupeKey)). Ask-gated (needs user approval; headless approval=never deployments auto-deny).knowledge_search — trigram OR + BM25, top-N; L1 query expansion by default (one small model call per query, cached per normalized query, timeoutMs fallback to lexical with degraded: 'lexical' metadata — never a tool error). expand: false or caller variants skips the call.knowledge_update — correction path; preserves id/provenance/stamps; scope/dedupeKey immutable after write.knowledge_list — auditability, cursor paging (newest first), expired/deleted flags.knowledge_delete — tombstone; expired items return false.- Events —
knowledge/written / knowledge/updated / knowledge/deleted with { id, scope, importance }.
knowledge_probe — evaluation-suite harness (stage-3 gates) that seeds the experiment corpus and reports arm recall + contract checks.
Authorization & trust (V1.11)
- Workspace isolation — identity is the caller's canonical cwd (
session.header.cwd); cross-workspace calls never see content.
- Ask gating —
knowledge_write/knowledge_delete go through the tools approval seam.
- Global scope —
allowedGlobalWriters allowlist (agent/preset ids); headless denial is explicit write-rejected.
authorTier — human | explicit | derived | llm, stamped by the tool layer (never caller input; spoofing → write-rejected). Hits carry provenance: original | enriched and authorTier.- Session scope — scratch memory visible only to the owning session.
Configuration
Plugin row config (all optional, defaults are the V1.11 proposal defaults):
# cordis.patch.yml / profile row
- id: knowledge-sqlite
name: 'dsh-knowledge-sqlite'
config:
gating: 'ask' # 'ask'（默认，V1.11）：write/delete 经 approval 确认；'none'：跳过门控自动放行
databasePath: '/custom/data/knowledge.sqlite' # 默认 $DSH_HOME/knowledge.sqlite（DSH_HOME 缺省 ~/.dsh）
maxContentTokens: 2048
queryExpansion:
enabled: true
model: '' # unset = session default model
maxOutputTokens: 300
timeoutMs: 3000 # 默认值（0.1.5 起）
cache: true
retrieval:
topN: 20
maxQueryTrigrams: 0 # 0 = unlimited; cap 24 needs idf-priority truncation (see EXPERIMENTS)
maxCandidates: 200
authorization:
allowedGlobalWriters: [] # agent/preset ids allowed to write global scope
corpusPath: 'research/memory-knowledge-seam/experiments/memory-experiment' # probe corpus base (relative to workspace)
Storage layout
- Single SQLite file (
items + two FTS5 trigram tables: items_fts_base = content only, items_fts_rich = content + enrichment columns joined).
- Upsert uses
INSERT … ON CONFLICT DO UPDATE (rowid stays stable so FTS links survive — INSERT OR REPLACE breaks them).
- Search filters
deleted/TTL/workspace at query time; ranking is bm25().
dedupeKey uniqueness = (workspace_id, scope, dedupe_key) (non-partial unique index; SQLite NULLs don't conflict).
Development
npm install
npm run build # tsc（纯 TS，无需装饰器转换）
npm test # vitest：38 项（store 19 + service 19）
node test-smoke.mjs # SQLite 层冒烟：复现 A 7% / C 21% / D 50% / human-A 65% + 契约检查
测试覆盖（提案测试矩阵裁剪）：契约往返、dedupeKey 作用域 upsert（确定性 id）、打标伪造拒绝、
global allowlist、scope/dedupeKey 不可变、TTL（过滤/list 标记/过期 delete=false）、tombstone、
跨 workspace / session / global 隔离、missing-key、closed、maxContentTokens、分页、富化字段检索、
L1 扩展（mock llm：命中/缓存/超时降级/失败降级）、budget 截断、事件发射、ask 门控监听器、
probe 实验套件（A/C/D 确定性臂精确复现 + contract 全过）。
Design & evidence
- DESIGN — V1.11 proposal full text (community design, not official)
- REVIEWS — 11-round review chain summary (0C/0M/0m)
- EXPERIMENTS — RESULTS-v3 + prototype verification (kpro-2, live L1, contract 11/11, ask-gate)
- Prototype workbench:
research/memory-knowledge-seam/experiments/prototype/ (SQLite blueprint, JS↔FTS5 calibration 0/72 mismatch, dynamic-plugin verification)
⭐ Support
If this plugin helps you, consider giving it a ⭐ Star on GitHub — it keeps the project going. Issues and PRs are welcome.
License
MIT © ninjasln
