dsh-web-fetch-moli
Moli（超轻量级 Rust 无头浏览器）Web 获取提供程序，适用于 DeepSeek Harness：渲染复杂的 SPA、微前端和静态页面，内存占用约 60MB。
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add github:GreyRaphael/dsh-web-fetch-moli#89ac9ebfaf7dfd7d48a90d805959101585c36300Web Fetch Moli 以 dsh-web-fetch-moli 发布，当前版本为 0.4.1。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
A DeepSeek Harness (DSH) plugin providing a Moli backend for the web_fetch tool. Built on Moli (an ultra-lightweight Rust headless browser), it renders dynamic micro-frontends, single-page applications (SPAs), and static sites with ~60MB memory footprint (>90% reduction compared to ~1GB for standard Chromium/Playwright), denoises pages via LinkeDOM + Readability + mdream, and returns clean, LLM-optimized GitHub Flavored Markdown.
Page.setBypassCSP), ignores certificate errors, and bypasses Service Worker caching. Drives native Moli layout tree computation via CDP Micro-Clip layout materialization (Page.captureScreenshot 1x1 micro-viewport) and triggers W3C standard scrollIntoView({ block: 'nearest' }) without any user-script monkey-patching, cleanly rendering complex micro-frontends (Alibaba Alfa, qiankun, single-spa) where other lightweight engines fail.local (default): Automatically manages a local moli serve daemon over CDP, providing full SPA and micro-frontend execution with ~60MB memory footprint.cdp: Connects to an existing remote Moli or Chromium CDP service over Chrome DevTools Protocol.cf-mitigated: challenge interstitials and waits in the same page/context for natural clearance without artificial bot behavior.| Feature / Metric | Chromium / Playwright | Lightpanda (Zig) | Obscura (Rust) | Moli (Rust) + DSH Plugin |
|---|---|---|---|---|
| Memory Footprint | ~800 MB – 1.2 GB | ~40 MB | ~50 MB | ~50 – 60 MB |
| Micro-Frontend Sandbox | ✅ Full | ❌ Crashes on iframe sandbox (contentWindow.bind) | ❌ Fails on dynamic style-loader | ✅ 100% (CDP + CSP bypass) |
| Infinite Scroll / Sentinels | ✅ Pixel layout loop | ❌ Geometry incomplete | ❌ Geometry incomplete | ✅ Native Layout Materialization (Micro-Clip CDP) |
| Execution Protocol | Heavy CDP | Lightweight CDP | Custom / CDP | Lightweight Native CDP |
web_fetch (tool-web)
└─ ctx.web.fetchProvider = moli
├─ local: resolves moli binary → manages `moli serve` daemon → connectOverCDP
│ └─ setupPageHooks: Page.setBypassCSP + ignoreCertErrors + bypassServiceWorker
├─ cdp: connectOverCDP(remoteEndpoint)
├─ page.goto → Micro-Clip layout materialization + native scroll loop → settle (networkidle) → page.content()
├─ denoise: LinkeDOM (DOM) → Readability (article extract) → mdream (Markdown)
└─ Markdown (or raw HTML when denoise is disabled)
dsh web), Node.js ≥ 20.moli binary is detected on your system, it automatically downloads and unpacks the latest release binary matching your OS and architecture (Linux, macOS, Windows x86_64 / arm64) into ~/.cache/moli.$PATH, ~/.local/bin/moli, or via the moliPath setting.Manual installation (optional):
curl -sSL https://github.com/GreyRaphael/moli/releases/latest/download/moli-x86_64-unknown-linux-gnu.tar.gz | tar -xz -C ~/.local/bin
chmod +x ~/.local/bin/moli
# Option 1: Install directly via GitHub repository
dsh plugin --profile web add github:GreyRaphael/dsh-web-fetch-moli
# Option 2: Install via npm package name
dsh plugin --profile web add dsh-web-fetch-moli
After adding, restart dsh web to load the plugin.
The settings card (设置 → 插件 → 插件配置 → Moli 网页爬取) configures web-fetch-moli in real time:
| Field | Default | Description |
|---|---|---|
backend | local | local (managed Moli CDP daemon) or cdp (remote CDP endpoint). |
moliPath | (auto) | Path to moli binary. Blank = auto-discover on $PATH and standard locations. |
cdpEndpoint | 127.0.0.1:9222 | Remote CDP endpoint (host:port or URL, used in cdp mode). |
bypassCsp | true | Bypasses Content Security Policy via CDP Page.setBypassCSP, unlocking micro-frontend dynamic script loading. |
autoScrollSentinel | true | Programmatically triggers IntersectionObserver load-more sentinels for structure-first geometry. |
shareBrowserContext | true | Preserves cookies and localStorage across fetches in cdp mode. Unchecked (and always in local mode): fresh isolated context per fetch. |
denoise | true | Runs LinkeDOM + Readability + mdream to strip ads, navbars, and footers before LLM-optimized Markdown conversion. |
maxConcurrency | (auto) | Max concurrent rendering slots (auto: local 20, remote CDP 50). |
challengeWaitMs | 15000 | Bounded wait (ms) for Cloudflare verification to clear naturally. |
challengeRetries | 1 | Same-page retry attempts after challenge wait. |
MIT