spa-api-extract
An Agent Skill that extracts structured data from JavaScript single-page apps (SPAs) when a normal page fetch fails.
When WebFetch (or any HTML-to-markdown fetcher) times out, returns only obfuscated/bundled JS, or returns an empty client-rendered shell, this skill switches to a reliable CLI pipeline that finds and calls the JSON API the page uses internally.
When to use
- The target page renders content client-side (React/Vue/Next/Nuxt SPA).
- The real data lives behind a JSON API, not in the initial HTML.
- The URL carries an auth token query param (e.g.
?t=xxxx).
WebFetch timed out or returned scripts instead of readable content.
The pipeline
- Confirm the failure signal — timeout / JS-only / empty shell → it's an SPA.
- curl-probe headers + raw HTML skeleton (
-sS -m always).
- grep the HTML/JS for
fetch() / /api/ endpoints, <script src>, and BASE-path derivation.
- Reconstruct & request the JSON API, carrying the token param (read-only
GET).
- Parse with Python — inspect structure first, then extract to a UTF-8 file (avoids Windows GBK mojibake).
- Verify record count, field completeness, non-empty output.
See SKILL.md for the full step-by-step commands and reference.md for the grep-regex library, common SPA data patterns (REST / GraphQL / __NEXT_DATA__ / inline window.__* JSON), auth variants, and a troubleshooting table.
Install
Drop the directory into your agent's skills folder, e.g.:
~/.qwenworkcn/skills/spa-api-extract/
The skill is plain Markdown + standard curl/python and follows the open Agent Skills convention (SKILL.md + frontmatter), so it is portable across harnesses that support the format.
DeepSeek Harness (dsh) plugin
This repo is also packaged as a DeepSeek Harness (dsh) skill plugin — dsh's model is "everything is a plugin", built on the Cordis runtime.
Layout:
package.json # dsh manifest: `dsh.bundle.patch` + `dsh-plugin` keywords
cordis.patch.yml # Cordis patch that inserts this plugin into the runtime tree
index.js # entry: inject = ['skills'], apply(ctx) -> registerSkills(ctx)
lib/skills.js # registers the root SKILL.md as one ctx.skills provider
SKILL.md # the skill bundle (at the package root)
reference.md # deep-dive loaded on demand
.skill-metadata.yaml # bilingual recommended queries
Install into a dsh deployment (after @deepseek-ai/dsh-base, which mounts the skill registry):
dsh plugin --profile web add github:sunwindy91/spa-api-extract
# （仓库自带 cordis.patch.yml，dsh 安装后会自动挂载；无需手工 patch）
The plugin registers knowledge only — it exposes no host tools and performs no network I/O at load time.
Note: the dsh ecosystem commonly nests skills under skills/<name>/SKILL.md. This plugin keeps the single bundle at the package root so the same tree also loads as a native QwenWork skill without duplicating the file; lib/skills.js reads it from there. The provider/candidate/definition shapes and frontmatter parsing match the official dsh skill providers.
Safety & compliance
- Read-only by default. Only
GET is used; write/mutate endpoints require explicit user authorization.
- Tokens are secrets. Captured tokens must not be echoed into final output, committed logs, or shared files.
- Respect the target. Only scrape sites/APIs you are authorized to access, and honor their Terms of Service, robots directives, and rate limits.
- No telemetry. This skill contains no listener, no data collection, and no network callback — it is pure instruction text with placeholder examples (
HOST / PATH / TOKEN).
License
MIT © 2026 sunwindy91 — see LICENSE.
安装提示（实测）：这条命令会把参数转发给 profile 目录里的 pnpm，安装需要能访问 github.com。
若本机配了代理但代理没在运行，pnpm 会继承 git 的代理设置并报 ECONNREFUSED ——
此时确保代理在跑，或临时清掉代理配置（git config --global --unset http.proxy）。
依赖说明：yaml 是 peerDependency（宿主已自带）；若环境里没有，插件会退化为内置的极简 frontmatter 解析器，
不会因为缺依赖而加载失败。
