dsh-context-milvus
English | 简体中文
A DSH plugin that provides semantic code search over a Milvus vector database, with a complete index ↔ search pipeline.
dsh-context-milvus = equips your DSH Agent with a dedicated codebase semantic search engine. Milvus handles high-speed vector retrieval, transforming "needle-in-a-haystack grep" into precise recall of relevant code snippets — reducing tokens, minimizing tool calls, and improving coding agent quality on large repositories.
Why dsh-context-milvus?
dsh-context-milvus is an open-source code semantic search plugin for DeepSeek Harness (DSH) coding agents, built on Milvus as the vector database and registered as a Cordis Plugin. Its core purpose: solve the high token consumption, excessive tool calls, context pollution, and poor large-codebase comprehension that plague native DSH Agent grep workflows.
Native DSH Agent workflow: encounter a problem → repeatedly search_code (grep) → read files → search again, flooding the prompt with irrelevant text, exploding tool calls, increasing token costs, and making it easy to miss dependencies in large repositories.
Solving the key pain points of native grep search
| Native grep workflow pain point | dsh-context-milvus solution |
|---|
| Literal string matching only — semantically related but differently named code is missed | Vector semantic search — matches by code meaning, not just keywords |
| Multiple tool call rounds, reading many irrelevant files, token explosion | Returns only truly relevant code snippets, split by AST at function/class boundaries for precision |
| Flooding context with grep output and irrelevant source code, causing context pollution and degraded model reasoning | Milvus pre-built index, Agent gets concise effective context in one tool call without search noise in the prompt |
| Thousands of files in large repos, Agent traversal is extremely inefficient | Milvus vector DB enables fast retrieval over millions of code blocks, supports incremental index updates without full repo rescanning |
| Can only search already-open or known-path files | After full-repo indexing, can semantically search any code location regardless of file path knowledge |
Features
search_code — Semantic code search: natural language query, returns matching code snippetsindex_code — Index codebase: AST parsing + chunking → Embedding → Milvus storageindex_status — View index status: file count, last index time, hash statisticsfind_callers — Code relationship analysis (impact analysis): find all references to a symbol, with cross-file import resolutiontrace_call_chain — Call chain tracing: BFS expansion from entry symbol (impact/dependency analysis), with cross-file resolution disambiguation- Hybrid search — BM25 keyword + vector semantic dual-path retrieval, RRF fusion,
hybridMode toggle
- Chunk overlap — AST chunks include surrounding context lines (
chunkContextLines, default 2) for better recall
- Query expansion — expands natural-language queries with code synonyms before embedding (
queryExpansion, default on)
- Two-stage reranking — retrieves a topK×3 pool then applies proportional term-overlap & name-match boosts, keeping the Milvus score primary (
rerankEnabled, default on)
- Ignore pattern system — Three-layer gitignore-style ignore rules (default + codebase + global)
- Incremental indexing — Merkle SHA-256 hash tracking, processes only changed files
- Workspace isolation — Independent Merkle state files per workspace, no interference
- ADR decision memory system — Records design rationale behind code changes (Architecture Decision Records), supports semantic search, CRUD, constraint injection, and consistency checking
- Code relationship analysis — Extracts symbol references from AST during indexing (
references, language-specific syntax nodes), supports cross-file exact matching
- Cross-file import resolution (V2) — Scans import/export statements using tree-sitter AST during indexing, builds a persistent bidirectional Import Map, enabling
find_callers/trace_call_chain to perform precise cross-file symbol matching (same-name disambiguation, cross-module tracing)
- Native telemetry (opt-in) —
search_code / index_code / index_status write one JSONL line per execution (disabled by default, no source code captured), with an analysis script for descriptive stats + Bootstrap CI + correlation
Effectiveness Evaluation
A reproducible statistical evaluation suite (see scripts/eval/) quantifies how dsh-context-milvus improves retrieval quality and end-to-end agent efficiency. It covers offline retrieval quality, end-to-end agent evaluation, and native telemetry — using nonparametric statistics (Wilcoxon, Bootstrap CI, Cliff's Δ) with a unified file-level relevance standard. Run instructions and full reports live in scripts/eval/*/output/report.md.
Offline retrieval quality — 21 annotated queries × 19-file multi-language corpus
Three retrieval strategies compared: G (grep keyword), R (naive RAG: sliding-window + pure vector), P (plugin: AST chunking + BM25 hybrid + RRF + chunk overlap + query expansion + two-stage reranking).
| Metric | G (grep) | R (naive RAG) | P (plugin) |
|---|
| recall@10 | 0.9524 | 1.0000 | 0.9524 |
| MRR | 0.6754 | 0.9524 | 0.8452 |
| nDCG@10 | 0.7446 | 0.9610 | 0.8725 |
| hit@1 | 0.4762 | 0.9048 | 0.7619 |
| precision@10 (file-level) | 0.4203 | 0.1095 | 0.2730 |
| precision@10 (chunk-level) | — | — | 0.3619 |
Note on precision@10: Two metrics are reported. File-level precision@10 de-duplicates results by file path (each file counted once), measuring how many unique relevant files appear in the top-K. Chunk-level precision@10 counts each result independently, matching the classic IR definition: "of the 10 entries the Agent sees, how many are from relevant files?" The chunk-level metric is higher because the Agent benefits from multiple chunks of the same relevant file clustering in the top results.
Key findings:
- Two-stage reranking lifts hit@1 by 6.7% (from 0.714 to 0.762 vs P0 baseline): proportional term-overlap (+30%) and name-matching (+15%) boosts improve first-hit accuracy. The gap to naive RAG (0.905) narrowed from 0.14 to 0.14.
- Chunk-level precision@10 = 0.362: the Agent sees ~3.6 relevant entries per 10 results. This is bounded by the corpus (each query has only 1-2 relevant files, each producing few chunks) — the ceiling is determined by the number of chunks per relevant file, not search quality.
- AST chunking + query expansion + chunk overlap drive precision: P vs R precision@10 +0.1635 (p=0.00013, Cliff's Δ=0.868 — large effect). Function/class-boundary chunks with surrounding context lines are far more focused than fixed sliding windows.
- Semantic search beats keyword grep on ranking: P vs G MRR +0.1698 (p=0.108), nDCG@10 +0.1280 (p=0.100) — relevant files rank higher, with near-significant p-values.
- grep precision is high but recall is brittle: G has the best precision@10 (0.4203) but the worst hit@1 (0.4762) — keyword-only search misses semantically-related code (e.g. "retry with exponential backoff" never matches
withRetry).
End-to-end agent evaluation — 8 tasks × 3 runs × 3 strategies
| Group | Average pass rate | Token consumption |
|---|
| G (grep) | 37.5% | baseline |
| R (naive RAG) | 50.0% | −928 vs G |
| P (plugin) | 62.5% | −2109 vs G |
Key findings:
- Highest task pass rate: P 62.5% vs G 37.5% vs R 50.0%.
- Significant token reduction: P vs G Δmean −2109 tokens/task (95% CI [−2325, −1864]), Wilcoxon p=0.014, significant after Holm correction, Cliff's Δ=−1.0. P also beats R by −928 tokens/task (p=0.014).
Native telemetry (opt-in)
search_code / index_code / index_status record execution metrics (query, result count, top score, duration, files/chunks indexed, etc.) as one JSONL line per call — disabled by default (telemetryEnabled: false), no source code content captured. Run node scripts/eval/telemetry/run.mjs to generate a descriptive statistics + Bootstrap CI + correlation report from ~/.milvus-index/telemetry.jsonl.
What role does Milvus play, and why Milvus?
- Stores AST-chunked code vectors: dsh-context-milvus uses tree-sitter AST to split code at function/class/method boundaries, generates embeddings, and stores them in Milvus — avoiding cutting a function in half.
- High-performance vector search: Encodes the query and performs vector similarity search with low latency, suitable for real-time Agent tool calls.
Note: BM25 keyword fusion is already implemented — Milvus native BM25 full-text search + vector semantic dual-path retrieval, RRF fusion (hybridMode enabled by default).
- Supports self-hosted Milvus / Zilliz Cloud, two deployment options; teams can control data; supports incremental indexing after code changes without full rebuild.
- Specifically adapted for code RAG: Supports path-scoped filtering (
search_code path parameter), allowing directory-limited searches — ideal for codebase scenarios.
DSH plugin architecture advantages
It is not a standalone MCP service, but a DSH plugin (Cordis Plugin) embedded directly into the DSH Agent process:
- Zero network overhead: Plugin and Agent share the same process, tool calls don't go through HTTP, latency far below MCP
- Naturally shares DSH resource configuration: Reuses DSH's config management, environment variable injection, and logging system — no additional configuration needed
- DSH Web GUI integration: Visual configuration through Settings → Plugins interface, no YAML hand-editing
- DSH ecosystem compatibility: Shares the tool registry with other DSH plugins (bash, agent-loop, web-search, etc.), Agents can freely combine them
Core Workflow
Registered DSH tools
| Tool | Function | Key Parameters |
|---|
search_code | Semantic code search | query (natural language), topK (result count), path (search scope) |
index_code | Index codebase | mode (full/incremental), path (target path) |
index_status | View index status | path (view per-workspace status) |
search_adr | Semantic ADR search | query (natural language), status, topK |
search_adr_by_file | Find ADRs by file path | file_path (code file path), status |
create_adr | Create new ADR | title (required), requirement, change_type |
update_adr | Update existing ADR | adr_id (required), content, status |
list_adrs | List ADR records | status, change_type, limit |
load_constraints | Load active ADR constraints | adr_ids, format |
check_adr_consistency | Check ADR-code consistency | file_path, fix |
find_callers | Find all references to a symbol for impact analysis, supports cross-file import resolution | symbol (required), direction, maxResults, sourceFile, resolve |
trace_call_chain | BFS call chain tracing from entry symbol (impact/dependency analysis), supports import resolution disambiguation | entry (required), direction, maxDepth, maxResults, resolve |
Workflow
- Run
index_code: Parse the project, split code blocks via tree-sitter AST → call Embedding API to generate vectors → store in Milvus collection.
- Agent encounters a coding problem, calls
search_code for hybrid search (vector semantic + BM25 keyword, RRF fusion).
- Milvus returns the most relevant code snippets, injected into the Agent's context.
- Agent debugs, refactors, or develops based on precise context — no more frantic grep file reading.
- After code changes, run
index_code mode=incremental to incrementally re-index only changed files.
- Check index status anytime with
index_status (indexed files, total code blocks, last index time).
- Before modifying code, use
find_callers for impact analysis: see which places reference the symbol to avoid missing cascading effects. For same-name symbols across files, use the sourceFile parameter to disambiguate by definition file.
- Understand call chains with
trace_call_chain: BFS expansion from entry function, direction=backward traces callers, direction=forward traces downstream dependencies. resolve: false falls back to V1 name-matching mode.
- Cross-file reference analysis:
find_callers and trace_call_chain enable import resolution by default (resolve: true). The Import Map built during indexing automatically maps import { X } from './foo' to foo.ts's exports, eliminating same-name ambiguity and supporting cross-module call chain tracing. Falls back to V1 name matching when the import map is not built.
ADR decision memory workflow
The ADR decision memory system records the "why" behind code changes (design decisions, trade-offs, constraints), enabling the Agent to not only read code but understand its evolution:
Note: ADR functionality is disabled by default. To enable it, set adrEnabled: true in the DSH config panel (Settings → Plugins → dsh-context-milvus).
- Before modifying code with ADR coverage, use
search_adr_by_file to check if the file has decision records, avoiding violation of existing decisions.
- When making design decisions, use
create_adr to record the context, alternatives, and rationale, and use update_adr to maintain code_anchors linking to code locations.
- When needing to understand constraints, use
load_constraints to load active ADR constraints into the context.
- After creating or updating ADRs, use
check_adr_consistency to verify ADR-code consistency, with fix for auto-repair.
- Use
search_adr for semantic search of historical decisions, understanding "why this was done this way."
Spec Document Fusion
When the brainstorming skill produces specification documents, they can be linked to the codebase through the following steps:
- Write spec documents: brainstorming output saved to
docs/superpowers/specs/YYYY-MM-DD-<topic>-design.md
- Generate anchors: Call
index_specs to automatically detect code references in the document and generate frontmatter + code_anchors
- Index:
index_code automatically scans docs/superpowers/specs/ and docs/superpowers/plans/ directories
- Discover:
search_adr returns both ADR and spec document results (with docType annotation)
Configuration
| Field | Default | Description |
|---|
specRoot | docs/superpowers/specs | Spec document directory (relative to indexRoot) |
planRoot | docs/superpowers/plans | Implementation plan directory (relative to indexRoot) |
Spec document fusion follows the adrEnabled toggle — no additional configuration needed.
Prerequisites
1. Install Ollama (Embedding service)
# macOS
brew install ollama
# Linux
curl -fsSL https://ollama.com/install.sh | sh
# Start Ollama service
ollama serve
Or use any OpenAI-compatible Embedding API service (OpenAI, Alibaba Cloud Bailian, etc.) by configuring embeddingEndpoint and embeddingApiKey.
2. Install Embedding model
# Pull nomic-embed-text model (default)
ollama pull nomic-embed-text
# Or other supported Embedding models:
ollama pull bge-m3
ollama pull mxbai-embed-large
3. Install Milvus (vector database)
Docker (recommended):
# Pull and start Milvus standalone
docker run -d --name milvus \
-p 19530:19530 \
-p 9091:9091 \
milvusdb/milvus:latest
# Verify connection
docker ps | grep milvus
Milvus cluster mode (Docker Compose):
# Download docker-compose file
wget https://github.com/milvus-io/milvus/releases/latest/download/milvus-standalone-docker-compose.yml -O docker-compose.yml
# Start
docker compose up -d
Or use Zilliz Cloud managed service — no self-hosting required.
Verify installation
# Verify Ollama
curl http://localhost:11434/api/tags
# Verify Milvus
docker run -it --rm \
-e MILVUS_URL=localhost:19530 \
milvusdb/milvus-sdk-node:latest \
node -e "const {MilvusClient} = require('@zilliz/milvus2-sdk-node'); \
new MilvusClient({address:'localhost:19530'}).listCollections().then(r=>console.log(r))"
Install to DSH
Method 1: From npm (recommended)
The plugin is published to the npm registry. Install directly via DSH CLI:
dsh plugin --profile web add dsh-context-milvus
The npm package includes pre-built dist/ output — no build step required during installation, avoiding the ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED error.
Method 2: From local tarball (offline / local development)
Build and package as a tarball, then install directly:
# 1. Build
npm run build
# 2. Package as tarball
pnpm pack
# 3. Install to profile
dsh plugin --profile web add ./dsh-context-milvus-0.1.3.tgz
pnpm pack produces a tarball containing the compiled dist/ output — no build step required during installation, so pnpm won't raise ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED.
Method 3: From Git (requires additional configuration)
dsh plugin --profile web add git+https://github.com/bobjia/dsh-context-milvus.git
dist/ output is not committed to git. The plugin uses the prepare script to automatically run tsc during installation.
pnpm 10 limitation: pnpm 10 blocks execution of build scripts by default. If you see:
ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED
The git-hosted package "dsh-context-milvus@0.1.2" needs to execute build scripts
but is not in the "onlyBuiltDependencies" allowlist.
Add to your profile's pnpm-workspace.yaml:
# ~/.dsh/profiles/<profile-name>/pnpm-workspace.yaml
onlyBuiltDependencies:
- dsh-context-milvus
Then re-run the install command. Or run pnpm approve-builds and select dsh-context-milvus.
To avoid this authorization, use Method 1 (npm) or Method 2 (tarball).
Configure the plugin
After installation, edit cordis.patch.yml under your profile:
# ~/.dsh/profiles/<profile-name>/cordis.patch.yml
- id: dsh-context-milvus
config:
milvusAddress: localhost:19530
milvusCollection: code_embeddings
milvusDim: 768
embeddingEndpoint: http://localhost:11434/api/embed
embeddingModel: nomic-embed-text
indexRoot: /path/to/your/code
indexExtensions: .ts,.tsx,.js,.py,.java,.go,.rs,.cpp,.cs,.scala,.php
hybridMode: true
bm25RrfK: 60
Restart DSH after configuration.
Build from source (local development)
If using a local development version:
1. Install dependencies
cd /mnt/home/bobjia/workspace/dsh-context-milvus
npm install --legacy-peer-deps
2. Create symlinks for @deepseek-ai packages
# Link DSH runtime packages (npm install may break these links)
ln -sf /mnt/home/bobjia/.npm-global/lib/node_modules/@deepseek-ai/dsh/node_modules/@deepseek-ai/cordis \
node_modules/@deepseek-ai/cordis
ln -sf /mnt/home/bobjia/.npm-global/lib/node_modules/@deepseek-ai/dsh/node_modules/@deepseek-ai/dsh-tools \
node_modules/@deepseek-ai/dsh-tools
ln -sf /mnt/home/bobjia/.npm-global/lib/node_modules/@deepseek-ai/dsh/node_modules/@deepseek-ai/schemastery \
node_modules/@deepseek-ai/schemastery
3. Register with DSH
# Install as local dependency
dsh plugin --profile web add file:/mnt/home/bobjia/workspace/dsh-context-milvus
dsh plugin add automatically adds the plugin to dsh.profile.bundles — no need to manually edit package.json.
4. Configure plugin
Edit ~/.dsh/profiles/<profile-name>/cordis.patch.yml (same as above) and restart DSH.
Configuration System
Priority (highest → lowest)
- Cordis Config (set via
cordis.patch.yml or DSH Web GUI)
- Environment variables (fallback)
- Defaults (e.g.,
localhost:19530)
Configuration fields
| Field | Environment Variable | Type | Default | Description |
|---|
milvusAddress | MILVUS_ADDRESS | string | localhost:19530 | Milvus server address |
milvusToken | MILVUS_TOKEN | string (secret) | empty | Milvus auth token |
milvusCollection | MILVUS_COLLECTION | string | code_embeddings | Collection name |
milvusDim | MILVUS_EMBEDDING_DIM | number | 768 | Vector dimension |
embeddingEndpoint | EMBEDDING_ENDPOINT | string | http://localhost:11434/api/embed | Embedding API URL |
embeddingApiKey | EMBEDDING_API_KEY | string (secret) | empty | Embedding API key |
embeddingModel | EMBEDDING_MODEL | string | nomic-embed-text | Embedding model name |
indexRoot | INDEX_ROOT | string | process.cwd() | Code repository root path |
indexExtensions | INDEX_EXTENSIONS | string | all supported extensions | File extensions to index (comma-separated) |
hybridMode | HYBRID_MODE | boolean | true | Enable hybrid search (BM25 full-text + vector semantic, RRF fusion) |
bm25RrfK | BM25_RRF_K | number | 60 | RRF fusion parameter k |
indexIgnoreDirs | INDEX_IGNORE_DIRS | string | dist, build, target, vendor, ... | Directories to skip during scan |
Tool Reference
search_code
Semantic code search. Automatically invoked when the user asks about code functionality, logic, or needs to find code by natural language.
Parameters:
| Parameter | Type | Required | Default | Description |
|---|
query | string | yes | — | Natural language query |
topK | number | no | 5 | Maximum results to return |
path | string | no | (configured root) | Path scope for search |
Return format:
[
{
"filePath": "src/auth/login.ts",
"content": "export async function loginUser(credentials) { ... }",
"score": 0.92,
"language": "typescript",
"chunkType": "function_declaration",
"name": "loginUser",
"startLine": 42,
"endLine": 68
}
]
index_code
Index the codebase. Supports two modes:
full — Full index of all filesincremental — Incremental index (only changed files, based on Merkle hash)
Parameters:
| Parameter | Type | Required | Default | Description |
|---|
mode | string | no | incremental | Index mode: full or incremental |
path | string | no | (configured root) | Path to index |
index_status
View index status, including file count, total code blocks, last index time, etc.
find_callers
Find all references to a symbol (function/variable/class) in the codebase, for impact analysis. V2 adds cross-file import resolution: use sourceFile to disambiguate same-name symbols across files.
Parameters:
| Parameter | Type | Required | Default | Description |
|---|
symbol | string | yes | — | Symbol name to find (function, variable, class) |
direction | string | no | backward | backward=who references me (impact); forward=who I reference (dependency) |
maxResults | number | no | 20 | Maximum results |
sourceFile | string | no | — | Definition file path (explicit disambiguation: only return callers that import from this file) |
resolve | boolean | no | true | Whether to enable import resolution (false falls back to V1 name-matching) |
Return format:
{
"chunks": [
{
"filePath": "src/auth/login.ts",
"content": "export async function loginUser(credentials) { ... }",
"startLine": 42,
"endLine": 68,
"chunkType": "function_declaration",
"name": "loginUser",
"resolution": {
"status": "resolved",
"targetFile": "src/auth/session.ts",
"exportedAs": "loginUser"
}
}
]
}
resolution field: status is resolved (resolved to a cross-file import), local (defined in the same file), or unresolved (fallback to V1 name-matching). Only present when import resolution is enabled and the Import Map is built.
trace_call_chain
Starting from the entry symbol, BFS-traverses the call chain along reference relationships. direction=backward for impact analysis (find who calls the entry), direction=forward for dependency analysis (what the entry calls). Uses a visited set to prevent cycles. V2 supports import resolution disambiguation (resolve: true by default), using filePath:symbol composite keys for cross-file call chain tracing.
Parameters:
| Parameter | Type | Required | Default | Description |
|---|
entry | string | yes | — | Entry symbol name |
direction | string | no | backward | Traversal direction |
maxDepth | number | no | 3 | Maximum recursion depth |
maxResults | number | no | 10 | Maximum results per level |
resolve | boolean | no | true | Whether to enable import resolution (false falls back to V1) |
Return format:
{
"chain": [
{
"depth": 0,
"symbol": "main",
"filePath": "src/index.ts",
"startLine": 1,
"endLine": 5,
"callers": ["runApp"]
},
{
"depth": 1,
"symbol": "runApp",
"filePath": "src/app.ts",
"startLine": 10,
"endLine": 20,
"callers": ["initConfig"]
}
]
}
Code Chunking
| Language | Extensions | Chunking method | Covered AST node types |
|---|
| TypeScript | .ts, .tsx, .mts, .cts | tree-sitter | function_declaration, method_definition, class_declaration, interface_declaration, enum_declaration, type_alias_declaration, arrow_function, generator_function, getter, setter |
| JavaScript | .js, .jsx, .mjs, .cjs | tree-sitter | function_declaration, method_definition, class_declaration, arrow_function, generator_function, getter, setter |
| Python | .py | tree-sitter + regex fallback | function_definition, class_definition, async_function_definition, decorated_definition |
| Java | .java | tree-sitter + regex fallback | class_declaration, interface_declaration, enum_declaration, method_declaration, constructor_declaration, record_declaration |
| Go | .go | tree-sitter + regex fallback | function_declaration, method_declaration, type_declaration, type_spec |
| Rust | .rs | tree-sitter + regex fallback | function_item, impl_item, trait_item, struct_item, enum_item, macro_definition |
| C++ | .cpp, .cxx, .cc, .hpp, .h, .hh | tree-sitter + regex fallback | function_definition, class_specifier, namespace_definition, struct_specifier, enum_specifier |
| C# | .cs | tree-sitter + regex fallback | method_declaration, class_declaration, interface_declaration, struct_declaration, enum_declaration |
| Scala | .scala | tree-sitter + regex fallback | class_definition, function_definition, trait_definition, object_definition, constructor_definition |
| PHP | .php | regex fallback | function_definition, class_declaration, interface_declaration, trait_declaration, enum_declaration |
All languages except PHP (regex-only) use tree-sitter AST parsing as the primary method. Python, Java, Go, Rust, C++, C#, and Scala automatically fall back to regex when tree-sitter parsing fails; TypeScript / JavaScript have no regex fallback — if tree-sitter parsing fails, the file is skipped (no index entry).
Ignore Pattern System (IgnoreMatcher)
Three-layer gitignore-style file ignore rules, ensuring only the code files that need analysis are indexed:
Three rule layers
- Built-in defaults: Automatically excludes
node_modules/, dist/, build/, .git/, __pycache__/, *.log, *.min.js, and 30+ common build artifacts and dependency directories
- Codebase ignore files: Automatically reads
.gitignore, .ignore, .xxxignore, etc. from the codebase root
- Global ignore file: Reads
~/.context/.contextignore (user-level global rules)
Automatic hidden path protection
Automatically ignores path segments starting with . (e.g., .git/, .vscode/, .env), preventing hidden directories and files from being indexed.
Backward compatibility
The indexIgnoreDirs config (comma-separated directory names) is automatically converted to gitignore-style patterns (e.g., dist → **/dist/**), maintaining compatibility with older versions.
Incremental Indexing & Workspace Isolation
Incremental Indexing (Merkle hash tracking)
- Uses SHA-256 hash tracking for each file's content changes
- Only re-indexes new or modified files; skips unchanged files
- Deleted files are automatically removed from Milvus
- State is persisted to a local JSON file
Workspace Isolation
- Different workspaces use independent Merkle state files
- State file paths are generated based on the workspace path's SHA-256 hash
- Indexing different workspaces does not interfere with each other
- The
path parameter in tool calls specifies the workspace, automatically using the corresponding state file
When to Use (and When Not To)
✅ Suitable Scenarios
- Codebases from tens of thousands to millions of lines, using DSH Agent for refactoring, bug localization, or cross-file reading
- Want to reduce token overhead and minimize Agent grep tool loops
- Need an open-source, self-hostable solution, avoiding closed-source indexing services
- Already using the DSH framework and want to enhance Agent code comprehension
- Need incremental indexing — code changes frequently but don't want full rebuilds every time
❌ Not Suitable / Caveats
- Requires an embedding API (OpenAI / Ollama, etc.), code snippets are sent to the embedding service during indexing; for high-privacy requirements, use Ollama local embeddings
- Adds Milvus / Zilliz Cloud as a dependency, increasing operational complexity; small codebases (a few hundred files) may not see significant benefit
- It is a retrieval augmentation tool, not a replacement for the model's context window — it filters high-quality context to solve "signal overload," not to infinitely expand the window
- Requires DSH environment (v0.6+), cannot run independently of DSH
Comparison: DIY Code RAG vs dsh-context-milvus
If you build your own code RAG for DSH Agent: you'd need to handle AST chunking, vector search tuning, incremental sync, DSH tool wrapping, result ranking, and ignore file systems. dsh-context-milvus packages all of this engineering into a plug-and-play solution, specifically tuned for code scenarios.
| Dimension | DIY Code RAG | dsh-context-milvus |
|---|
| AST Chunking | Integrate tree-sitter yourself, configure per language | Built-in 10-language tree-sitter chunking, auto fallback to regex |
| Semantic Search | Call embedding service and tune parameters yourself | Built-in vector semantic search, plug-and-play (BM25 keyword fusion) |
| Incremental Indexing | Implement file hash comparison and state management yourself | Built-in Merkle file state tracking, SHA-256, incremental updates |
| Workspace Isolation | Handle multi-workspace state conflicts yourself | Automatic path-hash-based isolation, no interference |
| Ignore Files | Implement .gitignore parsing yourself | Built-in three-layer ignore rule system (default + codebase + global) |
| DSH Tool Wrapping | Wrap DSH tools yourself (defineTool) | 13 native DSH tools (5 code tools + 8 ADR tools), one-click registration, formatted output |
| Configuration UI | Build yourself or hand-write YAML | DSH Web GUI visual configuration, 13 config fields |
| Config Sources | Single source | Three-source merge (Cordis Config > env vars > defaults) |
| Index Status | Build yourself | Built-in index_status tool, real-time index status |
DSH Web Configuration
After installation, go to the DSH Web interface (http://127.0.0.1:3080) Settings → Plugins to see dsh-context-milvus and its configuration form, supporting:
- Text inputs (standard fields)
- Password inputs (secret fields like
milvusToken, embeddingApiKey)
- Number inputs (number fields like
milvusDim)
- Toggles (boolean fields like
hybridMode)
- Field descriptions / help text
Architecture
┌───────────────────────────────────────────────────────────────────────────────────┐
│ DSH Agent / Web UI │
│ search_code │ index_code │ index_status │ find_callers │ trace_call_chain │
│ search_adr │ create_adr │ list_adrs │ load_constraints │
│ check_adr_consistency │
└───────────────────────────────────────────────────────────────────────────────────┘
│
┌───────────────────────────────────────────────────────────────────────────────────┐
│ dsh-context-milvus │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────┐ ┌──────────────────┐ │
│ │ chunker │→ │embedding │→ │ milvus │ │ merkle │ │ ADR module set │ │
│ │(AST+regex)│ │ client │ │ service │ │ tracker │ │ frontmatter/ │ │
│ └────┬─────┘ └──────────┘ └────┬─────┘ └────────────┘ │ chunker/anchor/ │ │
│ │ │ │ service/indexer/ │ │
│ ┌────▼───────────────────────────▼───┐ │ tools/constraint │ │
│ │ code-relations.ts (BFS engine) │ └──────────────────┘ │
│ │ findCallers / traceChain │ │
│ └────────────────────────────────────┘ │
│ ┌──────────────────────────────┐ ┌──────────────────────────────────────────┐ │
│ │ import-resolver.ts │ │ ignore-matcher (gitignore-style 3-layer) │ │
│ │ Import Map (persistent bi- │ │ ① DEFAULT_IGNORE_PATTERNS → ② codebase │ │
│ │ directional resolution) │ │ ③ ~/.context/.contextignore │ │
│ └──────────────────────────────┘ └──────────────────────────────────────────┘ │
└───────────────────────────────────────────────────────────────────────────────────┘
│
┌─────────┴─────────┐
│ │
┌──────────┐ ┌──────────┐
│ Milvus │ │Embedding │
│ (vector DB) │ API │
└──────────┘ └──────────┘
Module dependency graph
index.ts (entry point)
├── config.ts — Config resolution (Cordis config > env vars > defaults)
│ └── DEFAULT_IGNORE_PATTERNS — Built-in gitignore-style ignore rules
├── milvus-service.ts — Milvus vector DB client wrapper (CRUD, search)
│ └── embedding.ts — OpenAI-compatible Embedding API client
├── merkle.ts — SHA-256 hash tracker (incremental indexing, persisted to JSON)
├── tools.ts — DSH tool definitions, formatting, workspace-aware tracker creation
│ └── code-relations.ts — Code relationship analysis engine (BFS call chain + dedup)
│ └── import-resolver.ts — Cross-file Import Map (tree-sitter AST import/export scan)
├── ignore-matcher.ts — gitignore-style pattern matching (file exclusion)
└── indexer.ts — Indexing pipeline orchestration
└── chunker.ts — tree-sitter AST chunking + regex fallback (includes references extraction + language import/export config)
└── adr-frontmatter.ts — YAML frontmatter parsing
└── adr-chunker.ts — Markdown section chunking
└── adr-anchor-index.ts — code_anchors reverse index
└── adr-service.ts — ADR CRUD + state management
└── adr-indexer.ts — ADR indexing pipeline
└── adr-tools.ts — 8 ADR tools
└── constraint-injector.ts — System prompt injection + re-injection
Testing
# Run all tests
npm test
# Test coverage
npm run test:coverage
# Single test file
npx jest test/dsh-context-remdb.spec.ts
# Code relationship analysis tests
npx jest test/code-relations.spec.ts
# Cross-file Import Resolution tests
npx jest test/import-resolver.spec.ts
# ADR module tests
npx jest test/adr-frontmatter.spec.ts
npx jest test/adr-chunker.spec.ts
npx jest test/adr-anchor-index.spec.ts
npx jest test/adr-service.spec.ts
npx jest test/adr-indexer.spec.ts
npx jest test/adr-tools.spec.ts
npx jest test/constraint-injector.spec.ts
Development
# Build
npm run build
# Type check (no output)
npx tsc --noEmit
# Run tests (verbose)
node --experimental-vm-modules node_modules/.bin/jest --no-cache --verbose
Dependencies
- @zilliz/milvus2-sdk-node — Milvus Node.js SDK
ignore — gitignore-style pattern matchingtree-sitter — AST parsing enginetree-sitter-typescript — TypeScript/JSX grammartree-sitter-python — Python grammartree-sitter-java — Java grammartree-sitter-go — Go grammartree-sitter-rust — Rust grammartree-sitter-cpp — C++ grammartree-sitter-c-sharp — C# grammartree-sitter-scala — Scala grammar@deepseek-ai/cordis — DSH framework (provided by DSH runtime)@deepseek-ai/dsh-tools — DSH tool registration API (provided by DSH runtime)@deepseek-ai/schemastery — Config schema definition (provided by DSH runtime)
License
MIT
