dsh-plugin-mlquant-benchmark
用于复现 ml-quant-trading protocol v1 基准测试的 DeepSeek Harness 工具。
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add github:initial-d/dsh-plugin-mlquant-benchmark#a476baf1d4c092d2a05a3ed739a642274bc70c61Plugin Mlquant Benchmark 以 dsh-plugin-mlquant-benchmark 发布，当前版本为 0.1.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
DeepSeek Harness tools for reproducing the
initial-d/ml-quant-trading
protocol v1 CPU benchmark.
The point is narrow: make a DSH agent able to run the existing benchmark, read the machine-readable artifact, validate it against the benchmark protocol, and draft an issue-ready report. This plugin does not add a trading agent, does not call market data APIs, and does not configure any model provider.
ml-quant-trading is a good reproducibility target for agent harnesses:
Challenge: can DeepSeek Harness reproduce a quant benchmark end to end, preserve the evidence bundle, and avoid turning runtime numbers into alpha claims?
This package registers four DSH tools:
| Tool | Purpose |
|---|---|
mlquant_benchmark_v1_cpu | Run the fixed protocol v1 CPU benchmark and write artifacts/benchmark-v1.json. |
mlquant_read_benchmark_json | Read the JSON artifact and render a compact Markdown result table. |
mlquant_validate_benchmark_json | Check protocol v1 fields, expected cases, fixed parameters, and variance warnings. |
mlquant_draft_github_issue | Draft a DeepSeek Harness benchmark issue body from the JSON artifact. It does not post to GitHub. |
Install the package in a DeepSeek Harness profile or preset environment:
dsh plugin --profile web add github:initial-d/dsh-plugin-mlquant-benchmark
The package declares a dsh.bundle manifest that inserts:
- id: mlquant-benchmark
name: dsh-plugin-mlquant-benchmark
If you use a local checkout while developing, add the same row manually:
- id: mlquant-benchmark
name: file:/path/to/dsh-plugin-mlquant-benchmark
This package is intentionally not published to npm yet. GitHub distribution is enough for the first DSH-facing benchmark reports; npm can come later if there is real usage.
Read AGENTS.md, docs/benchmarking.md, and docs/reality_check.md.
Use the mlquant benchmark tools to run the protocol v1 CPU benchmark, validate
and read the JSON artifact, and draft a DeepSeek Harness benchmark report. Keep
the result as an engineering reproducibility benchmark, not a trading-performance
claim.
Post the drafted report through the main repository's dedicated template:
https://github.com/initial-d/ml-quant-trading/issues/new?template=deepseek_harness_benchmark.yml
Seed example:
https://github.com/initial-d/ml-quant-trading/issues/61
npm install
npm test
The test loads the plugin with a mock ctx.tools.register, verifies that the
four tools register, reads and validates sample artifacts, and drafts an issue
body.