dsh-plugin-android-vscreen
Let an agent work on your Android phone without taking over your screen.
让 AI 在安卓手机的虚拟显示器上干活 —— 手机你自己的屏幕照常用，互不打扰。
A DeepSeek Harness plugin.
Tested with DSH on Android; the adb transport also works with DSH on a computer.
Why
Most "agent drives my phone" plugins tap the main screen: your phone is hijacked while the agent
works — you can't use it, and it can't be used quietly in the background.
Android can create a virtual display: a second, invisible screen. The agent launches an app onto
that display, looks at it and taps it there, while the physical screen keeps doing whatever you were
doing. Every action reports a "main-screen proof" (mainBefore → mainAfter) so you can see the
physical screen was not touched.
⚠️ A virtual display isolates the picture, not the effects / 只隔离画面，不隔离副作用
An app running on a virtual display still acts on the whole device. Tap Airplane mode, Wi-Fi
or Do not disturb inside a virtual display and that switch really flips — it is the same system
setting. Installing, sending messages, changing settings: all real.
So "your screen stays untouched" is not the same as "your device stays untouched".
Rule for agents using this plugin: take a frame with android_vscreen_see first, and derive tap
coordinates from what you actually see. Never tap a guessed fraction of the screen.
Tools
| Tool | What it does |
|---|
android_vscreen_doctor | Can I drive this device, and over which transport. Run it first. |
android_vscreen_status | Is a virtual display running, and how big is it. |
android_vscreen_start | Create the virtual display and optionally launch an app onto it. |
android_vscreen_see | Grab a frame of the virtual display as a PNG. |
android_vscreen_tap | Tap on the virtual display. |
android_vscreen_swipe | Swipe on the virtual display. |
android_vscreen_close | Destroy the virtual display. |
How it reaches the phone
Two transports, picked automatically:
| Transport | When it's used |
|---|
adb | DSH runs on your computer (USB or wireless debugging) |
local bridge (127.0.0.1:3081) | DSH runs on the phone itself (e.g. the Android build) |
Both end up as a shell on the device — the same privilege level adb shell has. That privilege is
required: an app cannot put another app onto a virtual display with its own identity, so the
bundled core runs as shell via app_process. It uses only public Android APIs
(DisplayManager.createVirtualDisplay, ImageReader) plus documented reflection constants — no
MediaProjection, no screen-capture permission dialog.
Install
From npm (once published) or straight from this repository — dsh plugin forwards to pnpm, so both
specs work:
dsh plugin --profile web add dsh-plugin-android-vscreen
dsh plugin --profile web add github:guzhou079-arch/dsh-plugin-android-vscreen
Or from a clone you already have:
git clone https://github.com/guzhou079-arch/dsh-plugin-android-vscreen
dsh plugin --profile web add link:./dsh-plugin-android-vscreen
Then ask the agent to run android_vscreen_doctor: it reports which transport is available and
probes the device.
Requirements
- An Android device reachable by
adb (USB or adb connect), or DSH running on the phone with
its privileged shell bridge available.
- Node.js 20+ (whatever your DSH runs on).
Development
The device side is a small Java program in assets/vscreen-core.jar. To rebuild it from source, see
the Android port repository (android-app/src/com/deepseek/harness/vscreen/). Two smoke tests run
the whole chain without going through a model:
node test-m1.mjs # create → launch → see → tap → see → main-screen proof → close
node vs-safe.mjs start # same thing, but aimed at a harmless target
License
LGPL-3.0-only — see LICENSE and THIRD_PARTY_NOTICES.md.
The bundled assets/vscreen-core.jar descends from Operit
(LGPL-3.0), which is why this plugin is LGPL rather than MIT.
