🎧 dsh-audiogen
AI audio generation for DeepSeek Harness (DSH) — turn your DSH web GUI into an audio studio: text-to-speech, music, sound effects and voice design, from the sidebar panel or straight from the Agent.
English | 简体中文
Main panel
✨ Features
- Four generation modes: text-to-speech, music, sound effects, and voice design
- Multi-vendor channels in one place: OpenAI-compatible TTS, MiniMax, ElevenLabs, Stability AI, or any custom OpenAI-compatible / generic POST endpoint
- Per-channel model & voice catalogs with one-click discovery, display aliases, capability categories, and per-model advanced fields (duration, seed, steps, cfg_scale, loop, prompt influence, and ElevenLabs SFX output format split into format/sample rate/bitrate, combined host-side into the single
output_format, …)
- Model comparison: run the same prompt across 2–4 models at once with per-model parameter overrides — results are grouped side by side
- Prompt enhancement: rewrite a rough idea into a ready-to-generate description with an LLM (pick any model from Settings → Models; falls back to the agent default model)
- History with one-click restore: prompt, config, model set and the original audio come back into the panel — no regeneration, no extra cost
- Resource library: auto-save generated audio (or opt in per run), organized by type — voices / music / SFX / TTS — with search, tags, rename, category moves, and full provenance (channel, model, voice id, prompt, params snapshot). Reuse a voice or music bed instead of regenerating
- Agent tools:
generate_audio and search_audio_library, manage_audio_voices (vendor voice browsing/deletion + prompt-based voice recommendation + role voice casting), plus bundled session skills — the Agent can generate and find audio on demand
- Role voice casting: assign a primary voice (+ backups) to each character of a novel/game —
manage_audio_voices action=cast takes character profiles (JSON array/object or a text description structured first) and applies deterministic hard filters (gender/age/use_case strict; accent is a preference relaxed only when the strict pool is empty) per character; the Agent picks voices globally (no primary reuse across lead/major roles) and action=save_cast validates membership, auto-fills backups, flags reuse and persists the plan to ~/.dsh/dsh-audiogen/cast-selections.json; then TTS with the chosen voice_id (or design a custom voice first via generate_audio(mode=voice_design))
- Panel voice management: a «音色» entry in the studio's left mode row (next to TTS/music/SFX/voice-design) — browse/filter vendor voices (language/keyword/source + official ElevenLabs shared-voice filters), ask the agent default model to recommend voices for a natural-language requirement (e.g. «清亮甜美的少女音»), preview, delete account-owned voices (confirmed) and backfill the chosen
voice_id into the TTS form; every AI recommendation is recorded automatically (last 50, shared by panel and Agent) so you can revisit requirements/channels/reasons and reuse a voice later
- Keys stay local: API keys live in the local DSH settings document and generation is proxied by the local host; the browser and the Agent never touch plaintext credentials
📸 Screenshots
📦 Installation
The plugin is published on npm. DSH host (Node ≥ 20) required.
dsh plugin --profile web add dsh-audiogen
Local development install:
dsh plugin --profile web add /path/to/dsh-audiogen
Restart dsh web after install — the sidebar will show the AI Audio entry.
🚀 Quick start
- Open Settings → Plugins → AI Audio
- Add a channel: pick a preset provider (+ Add provider) or a custom endpoint (+ Add custom provider)
- Fill in the API URL, API key, and the model/voice catalog (use Fetch available models to import them)
- Save, then open the AI Audio sidebar panel:
- choose a mode (Speech / Music / Sound effects / Voice design)
- type your text or prompt (optional: ✨ Enhance prompt)
- pick a model — or tick Model comparison for 2–4 models at once
- press Start generation and play the results, download them, or add them to the resource library
🎛 Modes supported by each vendor
| Mode | MiniMax | ElevenLabs | Stability AI | OpenAI-compatible / custom |
|---|
| TTS | ✅ (8 voices) | ✅ (voices + streams) | — | ✅ |
| Music | ✅ (music-3.0 / music-2.6 / music-cover) | ✅ (music_v2) | ✅ (stable-audio-*) | ✅ (generic POST) |
| Sound effects | — | ✅ (eleven_text_to_sound_v2, loop / prompt influence / output format as codec+sample rate+bitrate → output_format) | ✅ (stable-audio-* — same text-to-audio protocol; auto-detected in both Music and SFX) | ✅ (generic POST) |
| Voice design | ✅ (/v1/voice_design) | ✅ (/v1/text-to-voice/design) | — | — |
🤖 Agent usage
| Tool | Purpose |
|---|
generate_audio | Submit a TTS / music / SFX / voice-design task; waits for completion and returns same-origin audio URLs. Optional enhance_prompt, save_to_library, per-vendor params. |
manage_audio_voices | Browse/filter the vendor voice libraries (MiniMax, ElevenLabs) with language/keyword/source filters; recommend top-k voices for a natural-language requirement (action=recommend, uses the agent default model, ids validated against the pool); role casting (action=cast prepares per-character filtered candidate pools from character profiles; action=save_cast validates + persists the plan); delete account-owned voices (official/shared/system voices are read-only and refused). Then use the returned voice_id with generate_audio (mode=tts). |
search_audio_library | Search the local resource library (type / category / keyword) and reuse an existing voice, music bed or effect. |
Typical session commands (skills bundled with the plugin):
/audio:tts Read this sentence with a warm voice
/audio:music Generate a 30-second lo-fi background track
/audio:sfx Create a sci-fi UI cue
/audio:design Craft a warm retro synth voice
🔐 Security & data notes
- API keys are stored in the local DSH settings document; requests are proxied by the local host (
/api/dsh-audiogen/*, loopback-only routes)
- Generation consumes your upstream provider quota; audio content is produced by the upstream model
- History & library persist under
~/.dsh/dsh-audiogen/
- Prompt enhancement calls the LLM model you choose (default: agent default model) — no extra API key
🛠 Development
pnpm install
pnpm run typecheck
pnpm run build # outputs lib/ (host + client bundles)
📄 License
Apache-2.0
