English · 简体中文
dsh-kb-manager — local knowledge base lifecycle for DeepSeek Harness: import, chunk, index, hybrid search
Import once. Retrieve with confidence.
dsh-kb-manager is a local knowledge-base lifecycle plugin for DeepSeek Harness (dsh): multi-format import → smart chunking → pure-TypeScript vector indexes → hybrid search (vector + BM25 → RRF → optional rerank) → citation tracing—plus snapshots/rollback, directory-watch sync, and portable .kbpack archives.
Ask in natural language. The plugin exposes 16 Agent tools and an optional Web panel (KB list, import wizard, search bench, source locator)—no separate RAG server required.
Why dsh-kb-manager?
| Capability | What it changes |
|---|
| Multi-format import | PDF (page-aware; scan pages flagged), DOCX, Markdown, HTML (Readability), CSV, JSON, TXT, plus URL fetch. |
| Smart chunking | fixed / recursive (default) / semantic; heading breadcrumbs, table-as-chunk, page/paragraph provenance. |
| Pure-TS indexes | Flat (exact) + HNSW (approx)—no native addons. |
| Hybrid retrieval | Dual-path recall → RRF (k=60) → optional rerank; debug: true returns per-stage scores. |
| Offline fallback chain | Local / OpenAI-compatible embedding → built-in HashEmbedder; rerank failure falls back to RRF order. |
| Snapshots & rollback | Manifest + index backup; embedding-model mismatch refuses unsafe restore. |
.kbpack portability | tar.gz of source + chunks + index + metadata; create_new / merge import. |
| Directory sync | chokidar + SHA-256 change detection + catch-up scan; deleted watch dirs pause without cascading deletes. |
| Read-only mode | read_only: true makes write tools return read_only_mode immediately. |
| Agent-friendly tools | Few-shot examples embedded in every tool description. |
Architecture
flowchart LR
A[Documents / URLs] --> B[Parse]
B --> C[Chunk]
C --> D[Embed]
D --> E[Vector index<br/>Flat / HNSW]
C --> F[BM25]
E --> G[Hybrid search]
F --> G
G --> H[RRF]
H --> I[Optional rerank]
I --> J[Citations + debug scores]
E --> K[Snapshots / .kbpack]
C --> K
Install
[!NOTE]
Requires an existing DeepSeek Harness installation.
From GitHub
dsh plugin --profile web add github:xiaoshi7915/dsh-kb-manager
On git install, the prepare script runs tsdown for a self-contained build (no monorepo context needed).
Build from source
git clone https://github.com/xiaoshi7915/dsh-kb-manager.git
cd dsh-kb-manager
npm install
npm run build
dsh plugin --profile web add .
Validate the composed profile, restart DSH, and refresh the Web UI:
dsh --profile web --dump-config
dsh web
Then try:
Create a knowledge base named “project-docs”, import this PDF, and search for how authentication works. Show me the source chunk.
How it works
- Create a KB (
create_kb) with an embedding model (default offline hash-embed-v1, or any OpenAI-compatible name when embedding_api_base is set).
- Import local files or URLs (
import_document) → parse → chunk → embed → index.
- Search with
search_kb / multi_kb_search: vector + BM25 → RRF → optional rerank; use debug: true to inspect stage scores.
- Trace hits with
get_chunk (surrounding context + provenance metadata).
- Optionally snapshot before risky edits, restore later, or ship a
.kbpack to another machine.
- Optional
auto_sync_dir watches a folder and incrementally updates the target KB.
Data lives under storage_path (default ~/.dsh/kb-manager/). The Web client reads the same service surface for overview, import, search, and source location.
Agent tools
| Tool | Description | Key params |
|---|
create_kb | Create a knowledge base | name*, description*, embedding_model?, tags? |
list_kbs | List all knowledge bases | — |
get_kb | KB details (model / docs / chunks / storage) | kb_id* |
delete_kb | Delete an entire knowledge base | kb_id* |
import_document | Import a local file or URL | kb_id*, source*, metadata? |
list_documents | List documents (optional status filter) | kb_id*, status_filter? |
delete_document | Soft-delete chunks; mark index dirty | kb_id*, doc_id* |
search_kb | Hybrid search; optional debug scores | kb_id*, query*, top_k?, filters?, rerank?, debug? |
multi_kb_search | Cross-KB search with source_kb | kb_ids*, query*, top_k? |
get_chunk | Chunk text + neighbors for tracing | kb_id*, chunk_id* |
rebuild_index | Full rebuild via shadow index swap | kb_id* |
create_snapshot | Create a version snapshot | kb_id*, note? |
restore_snapshot | Roll back to a snapshot | kb_id*, snapshot_id* |
export_kb | Export .kbpack or JSON | kb_id*, format?, output_path* |
Write tools (create_kb, delete_kb, import_document, delete_document, rebuild_index, create_snapshot, restore_snapshot, import_kb) return { success: false, error: 'read_only_mode' } when read-only. Domain errors are returned as { success: false, error: <code>, message } (not thrown).
When the Agent uses it
- “Put this PDF / doc / page into the knowledge base” →
import_document
- “Search the KB for X” →
search_kb
- “Find Y across all knowledge bases” →
multi_kb_search
- “Where did this citation come from?” →
get_chunk
- “Retrieval feels wrong—show stage scores” →
search_kb({ debug: true })
- “Back up before a big change” →
create_snapshot
- “Roll back to before the change” →
restore_snapshot
- “Pack this KB for another machine” →
export_kb + import_kb
- “Will local file edits sync?” → directory watch incremental sync
- “How big is this KB?” →
list_kbs / get_kb_stats
Configuration
Defaults work offline. Override in a trusted profile (id: kb-manager):
| Field | Default | Notes |
|---|
storage_path | ~/.dsh/kb-manager/ | Storage root; ~ expands to the home directory |
default_embedding_model | hash-embed-v1 | Offline default; or an OpenAI-compatible model name |
embedding_api_base | '' | OpenAI-compatible endpoint; empty → HashEmbedder |
embedding_api_key | '' | Embedding API key |
chunk_size | 512 | Chunk size (characters) |
chunk_overlap | 50 | Overlap length |
chunk_strategy | recursive | fixed / recursive / semantic |
top_k | 5 | Default hit count |
enable_rerank | true | Enable post-RRF rerank |
rerank_endpoint | '' | Remote rerank; empty → built-in rule reranker |
index_type | hnsw | hnsw / flat |
auto_sync_dir | '' | Watch path; empty disables |
auto_sync_kb_id | '' | Target KB for sync |
auto_sync_interval | 300 | Catch-up interval (seconds) |
max_file_size_mb | 100 | Per-file size cap (MB) |
read_only | false | Disable write tools |
Example:
- id: kb-manager
config:
storage_path: ~/.dsh/kb-manager/
default_embedding_model: text-embedding-3-small
embedding_api_base: https://api.openai.com/v1
chunk_strategy: recursive
index_type: hnsw
enable_rerank: true
read_only: false
Boundaries
- Reads/writes under
storage_path; import_document / import_kb also read user-specified paths or URLs; export_kb writes to output_path.
- Unsupported formats return
unsupported_format; oversized files are rejected by max_file_size_mb.
- Fully local by default; network only when
embedding_api_base / rerank_endpoint or URL import is configured.
export_kb does not mutate KB data and remains available in read-only mode.- Pure-TS indexes: no native vector DB process to operate.
Differentiating vs typical RAG stacks
| Capability | This plugin | Common RAGFlow / Dify / kotaemon / pdfkb-mcp setups |
|---|
| KB snapshots & rollback | ✅ | Usually missing |
.kbpack (source + chunks + index + metadata) | ✅ | Usually missing |
| Directory watch + catch-up scan | ✅ chokidar + SHA-256 | Partial / missing |
| Per-stage retrieval debug scores | ✅ vector / BM25 / RRF / rerank | Often opaque |
| Full offline embed fallback | ✅ HashEmbedder | Often needs external services |
| Read-only mode | ✅ | Rare |
Layout
dsh-kb-manager/
├── package.json cordis.patch.yml tsconfig.json tsdown.config.ts vitest.config.ts
├── awesome-entry.yml README.md README_ZH.md
├── assets/readme/ # hero.png (paste generated banner here)
├── src/
│ ├── index.ts # Plugin entry (name / inject / Config / apply)
│ ├── config.ts # Schemastery config schema
│ ├── core/ parse/ chunk/
│ ├── embed/ index/ search/
│ ├── kb/ # KBService, snapshots, sync, kbpack
│ ├── tools/ # 16 Agent tools
│ └── client/ # Web panel
└── tests/
Development
npm install # install deps (prepare builds automatically)
npm run build # tsdown dual entry → lib/index.js + lib/client.js
npm test # vitest
npm run typecheck # tsc --noEmit
License
MIT © 2026 xiaoshi7915
