dsh-android-agent — Android phone testing plugin for DSH
中文 | English
A mobile-app testing plugin for DeepSeek Harness: it connects over WebSocket to an Android phone
running dsh-android-agent on the same LAN and
lets you open apps, tap at coordinates, input text, swipe, send key events, take screenshots and
dump the UI tree from the dsh-web「Phone」panel, with the same phone_* tools exposed to agents.
Built entirely on official NPM SDK packages — no dsh source changes.
Features
| Feature | Description |
|---|
| Device management | CRUD phone devices (alias / WebSocket URL / token / notes); stored in ~/.dsh/dsh-phone.json (0600) |
| Connections | One WebSocket per device (ws://phone-ip:8080/ws?token=...), auto-reconnect (max 3), 5s status refresh |
| Open app | openApp by package name (e.g. com.android.settings) |
| Tap / swipe | tap/swipe at pixel coordinates, configurable gesture duration |
| Input text | inputText — optional tap-to-focus first (accessibility ACTION_SET_TEXT / paste based) |
| Key events | keyevent (back=4, home=3, recents=187, enter=66, delete=67 ...) |
| Screenshot | preview + download from the panel; agent tool saves into ~/.dsh/phone-screenshots |
| UI tree | getUI dumps accessibility nodes (text / class / bounds / actions) to locate elements |
| Agent tools | phone_list / phone_rpc / phone_open_app / phone_tap / phone_input / phone_swipe / phone_keyevent / phone_screenshot / phone_get_ui / phone_ui_find / phone_ui_tap / phone_ui_input / phone_ui_back — same device config as the GUI |
| UI-tree-driven actions | phone_ui_find (contains / resourceId) locates nodes in the accessibility tree and returns bounds + tap center; phone_ui_tap clicks the best matching node; phone_ui_input focuses a field by label and types; phone_ui_back sends the Android back key or the edge-swipe back gesture. The GUI panel now renders the UI tree as a clickable row list (tap / input buttons per node) — no manual coordinate typing needed. |
Security model
- All
/api/dsh-phone/* routes are loopback-only (with same-origin checks) — the execution
surface that drives a real phone is never exposed to the LAN.
- Tokens are stored in plaintext in
~/.dsh/dsh-phone.json (0700 dir / 0600 file, atomic writes) —
same trust model as dsh-ssh.
- Agents can only use devices configured by the user in the GUI first.
- Screenshots/actions consume real phone resources and battery; confirm before operating.
Phone prerequisites
- Install and start the Agent app's foreground service (enable accessibility / screen capture on the phone, same LAN).
- Note the Agent's WebSocket endpoint
ws://phone-ip:8080/ws and its token.
- Add the device in the dsh-web「Phone」panel: alias, WebSocket URL, token.
Install
This plugin is a standalone package (not part of the @linxin666/dsh-web-ui-all aggregate), installed via a local link:
### npm (not yet published — available once published)
dsh plugin --profile web add dsh-android-agent
### local standalone package (development, at D:\code\dsh\dsh-android-agent)
cd D:\code\dsh\dsh-android-agent
pnpm install && pnpm build
dsh plugin --profile web add link:D:\code\dsh\dsh-android-agent
Windows users can install with one command (build + link, PowerShell):
powershell -ExecutionPolicy Bypass -File D:\code\dsh\dsh-android-agent\scripts\install.ps1
Full install & verification checklist (build → tests → install → GUI/device checks) lives in
docs/VERIFY.md.
Restart dsh web afterwards: the「Phone」entry appears in the sidebar and the agent prompt picks
up the plugin announcement automatically.
Data
- Device config:
~/.dsh/dsh-phone.json
- Agent screenshots:
~/.dsh/phone-screenshots/
Development
pnpm install
pnpm typecheck
pnpm build
Known limitations
inputText relies on accessibility injection (ACTION_SET_TEXT / clipboard paste); some input
fields may reject it — under root/Shizuku you can fall back to adb-style input via shell.- Screenshots travel as base64 inside JSON-RPC; panel preview defaults to JPEG (PNG supported).
- Auto-reconnect can replay unfinished operations (non-idempotent) — mind side effects in long flows.
- The plugin only operates devices the user configured in the GUI.
