dsh-gemini-multimodal
DeepSeek Harness 插件：通过 Gemini API 或本地 Antigravity CLI 提供多模态工具（图像/音频/视频/文档理解、转录、图像生成）。
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add dsh-gemini-multimodal@0.1.2Gemini Multimodal 以 dsh-gemini-multimodal 发布，当前版本为 0.1.2。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
dsh-gemini-multimodal — eyes and ears for DeepSeek Harness
Gives DeepSeek Harness multimodal perception: the harness model (e.g. DeepSeek) is text-only, so this plugin delegates seeing/hearing to Gemini — image/audio/video/document understanding, transcription, and image generation.
DeepSeek models are text-only. When a task involves a screenshot, a chart, a voice memo, a video clip, or a PDF, this plugin lets the agent actually look and listen — and keep the judgment in the main model.
| provider | what it needs | when to use |
|---|---|---|
gemini_api | a Gemini API key from aistudio.google.com | default; direct REST, fast, one HTTP call |
antigravity_cli | local agy CLI, signed in with your Google account | no key to manage; uses your Antigravity/Google One quota |
Where to get the key (gemini_api): go to aistudio.google.com → sign in with your Google account → Get API key → Create API key. Keys start with AIza... or AQ. — both work. Pro/Google One subscribers get the same free-tier API quota (image generation is rate-limited on the free tier).
antigravity_cli setup: curl -fsSL https://antigravity.google/cli/install.sh | bash, then run agy once and sign in. No key needed.
dsh plugin --profile web add dsh-gemini-multimodal
Configure in your profile/settings layer (optional — zero-config: with no config at all it defaults to antigravity_cli and works if agy is installed):
- id: gemini-multimodal
name: dsh-gemini-multimodal
config:
provider: gemini_api # optional; default antigravity_cli
api_key: <your gemini key> # only for gemini_api
# output_dir: /path/for/images # generated images land here (default OS temp)
| tool | what it does |
|---|---|
media_understand | analyze an image / audio / video / URL (OCR, charts, UI, spoken words) |
media_transcribe | verbatim text of audio/video, timestamps where useful |
image_generate | text prompt → PNG saved locally (free-tier image quota is low; 429s possible) |
read_document | summarize or answer questions about PDF / Office / text files |
Headless agy -p cannot prompt for permissions, so the plugin passes --dangerously-skip-permissions by default. That grants the agent full tool access for that run. To tighten: set skip_permissions: false and add allow-rules in ~/.gemini/antigravity-cli/settings.json (e.g. read_file(...), deny shell). Prefer gemini_api when you need a narrower trust boundary.
gemini_api: media is sent inline (base64) to Google's Gemini API — nothing stored by the plugin, key lives in your config only.antigravity_cli: media is read by the local agent and sent to Google per the agy flow.npm install
npm run typecheck
npm test # mime mapping / agy args / smoke (no network)
npm run build
MIT