dsh-omni-modal
为 DeepSeek Harness 添加“全模态设置”部分：配置兼容 OpenAI 的图像/视频生成端点，并支持切换配置文件，同时向模型公开 generate_image / generate_video / check_video 工具。
插件会安装到这里；不确定时保持 web。
npx -y @deepseek-ai/dsh plugin --profile web add github:yzhclcl/dsh-omni-modal#caeddd8aa38802c8f8435368882775e1e909e765
Omni Modal 以 dsh-omni-modal 发布，当前版本为 0.1.0。Plugin Hub 会校验它的 manifest，并保存精确安装来源，便于复现安装结果。
Give a text-only model the ability to draw pictures and shoot videos — without touching a single line of DeepSeek Harness source.
dsh-omni-modal adds an Omni-modal settings section to the DSH settings sidebar, where you can point the harness at any OpenAI-compatible image / video endpoint and register three model-visible tools:
| Tool | What it does |
|---|---|
generate_image | Synchronous text-to-image. Saves the PNG/JPEG/WebP to disk and (when the attachment service is available) returns an inline preview card. |
generate_video | Asynchronous submit. Returns a task id immediately — the clip is still rendering. |
check_video | Poll that task; on success it downloads the finished video to disk. |
Status: usable, tracks the Harness developer preview — interfaces may change.
profile argument to use a non-active profile for a single call.baseUrl / apiKey / model / path / size for each side.{"quality":"high"}, {"mode":"text","aspect_ratio":"16:9"}, …).GET <baseUrl>/models (falling back to the base URL itself) without spending generation quota.Requires Node.js ≥ 18 and DeepSeek Harness.
dsh plugin --profile <profile> add github:yzhclcl/dsh-omni-modal
<profile> is your DSH profile name (web, desktop, …). Add #v0.1.0 to pin a tag.
Then restart DSH — the host half registers its routes and tools at boot, and the settings section appears under Settings → Omni-modal settings.
Copy this repository somewhere permanent, then add it to ~/.dsh/profiles/<profile>/package.json:
{
"dependencies": {
"dsh-omni-modal": "link:/absolute/path/to/dsh-omni-modal"
},
"dsh": {
"profile": {
"bundles": ["dsh-omni-modal"]
}
}
}
and mirror the same entry into the importers. → dependencies block of pnpm-lock.yaml, then create the link:
New-Item -ItemType SymbolicLink `
-Path "$HOME\.dsh\profiles\<profile>\node_modules\dsh-omni-modal" `
-Target "/absolute/path/to/dsh-omni-modal"
Never copy the package into
node_modulesinstead of linking it. DSH uses aSymbolas the cross-module call key; loading the module twice makes those symbols unequal and every tool call then throwsCannot read properties of undefined (reading 'prepare'), with no recovery after a restart.
Once installed, just talk to the agent:
generate_imagegenerate_video, then check_video| Section | Contents |
|---|---|
| Config profiles | Pick the active profile, rename it, add one (copied from the current) / delete one |
| Image | Base URL, API key, model, request path, default size, extra request params |
| Video | Base URL, API key, model, submit path, query path, default resolution, default duration, extra request params |
| Task records | Submitted video tasks with their status and artifact path |
Notes:
Saving takes effect immediately for the tools — no restart.
The base URL must be the address alone. Documentation examples are easy to copy verbatim as POST https://api.example.com/v1; the plugin strips a leading POST/GET/… automatically, but writing just https://api.example.com/v1 is safer.
The query path accepts two forms: a relative path (/videos/generations/{id}) appended to the base URL, or a full URL (https://api.example.com/agnesapi?video_id={id}&model_name={model}) used as-is. The full form is necessary when the query endpoint does not live under the base URL.
Placeholders available in the query path:
| Placeholder | Replaced with |
|---|---|
{id} | the task id returned by the submit call |
{model} | the model name configured in that profile |
The tools always use the active profile unless you pass profile explicitly.
generate_image(prompt, size?, model?, profile?, save_to?)
generate_video(prompt, model?, profile?, size?, seconds?, image_url?)
check_video(task_id, profile?, save_to?)
generate_image returns synchronously. The file lands in the plugin data directory's media/ and, if the attachment service is available, an image attachment is produced so the picture shows up in the conversation. The model itself cannot see the pixels — to actually look at the image, call read_image on the returned path.generate_video is asynchronous: it returns a taskId right away. Call check_video(task_id) later; when the status becomes succeeded the artifact is downloaded into media/ automatically.check_video failures are soft failures: a missing task (404), an unreachable download URL or a failed download return status: "failed" / "succeeded" plus a message, instead of throwing. These are expected business outcomes, and throwing would abort the whole tool-calling round.video_url / videoUrl / url, nested video.url, data.*, output.*, result.*, videos[0].url, and more.video_id / videoId over id / task_id — some services (Agnes, for instance) return an internal id that cannot be used for polling.Agnes AI's image and video endpoints have different shapes; the video side needs the official parameter set. Using apihub.agnes-ai.com:
Image
| Field | Value |
|---|---|
| Base URL | https://apihub.agnes-ai.com/v1 |
| Model | agnes-image-2.5-flash |
| Request path | /images/generations |
| Default size | e.g. 1312x736 |
Video
| Field | Value |
|---|---|
| Base URL | https://apihub.agnes-ai.com/v1 |
| Model | agnes-video-2.5-flash |
| Submit path | /videos |
| Query path | https://apihub.agnes-ai.com/agnesapi?video_id={id}&model_name={model} |
| Default resolution | 720P (Flash only accepts 720P; anything else returns 400) |
| Default duration | 5 (a string, range "4"–"12") |
| Extra request params | {"mode":"text","aspect_ratio":"16:9"} |
POST /v1/videos (not /v1/videos/generations), and mode is required (text / keyframe / reference) — put it in the extra params.GET /agnesapi?video_id=…&model_name=…, which is outside /v1, hence the full URL in the query path.aspect_ratio: 21:9→1680x720, 16:9→1280x704, 4:3→960x720, 1:1→720x720, 3:4→720x960, 9:16→720x1280.| What | Where |
|---|---|
| Configuration | ~/.dsh-omni-modal/config.json (contains the API key, mode 0600) |
| Generated artifacts | ~/.dsh-omni-modal/media/ |
| Video task records | ~/.dsh-omni-modal/tasks.json |
Set DSH_OMNI_DATA_DIR to relocate all of it (handy for isolated testing).
dsh plugin --profile <profile> remove dsh-omni-modal
For a manual install, drop the dependency and bundle entries plus the pnpm-lock.yaml entry, then remove the link:
cmd /c rmdir "$HOME\.dsh\profiles\<profile>\node_modules\dsh-omni-modal"
Removing a symbolic link must be done with
cmd /c rmdiror[IO.Directory]::Delete(path, $false); neverRemove-Item -Recurse, which follows the link and deletes the source directory. Quit DSH completely first.
~/.dsh-omni-modal/ holds your data — delete it yourself if you want it gone.
lib/index.js — the host half (Node ESM): config storage, the /omni-modal/* routes, and the three tool definitions.lib/client.js — the browser half (a hand-written CJS bundle loaded through window.__ModuleLoader__.load): registers the settings.section slot and renders the settings page.cordis.patch.yml — the plugin registry insert row, carried by dsh.bundle.patch.The host half imports no bare @deepseek-ai/* modules (third-party plugins cannot resolve those internal exports) — only node:* and relative paths. Tool parameter and output schemas are written as standard JSON Schema (required as a string array) rather than the defineTool property-map DSL, which would make the LLM API reject the whole function schema.
见 README.zh.md.