dsh-phone-control
Verifieddsh-phone-control · v0.1.14 · MIT · Web UI
Control an Android phone from DeepSeek Harness: read the accessibility tree, tap by text, type text, take screenshots, read and write files. Pair the companion app once — by code or by scanning a QR code.
Install
dsh plugin add dsh-phone-control Confirm the layer applied with dsh --profile default --dump-config — see the install guide.
Source
Tags
Readme
dsh-phone-control
A DeepSeek Harness plugin that drives a real Android phone: read the screen as an accessibility node tree, tap by text, type into fields, screenshot only when it helps.
Docs: this file (English) · 简体中文
Why
A 1080×2400 screenshot is 1–2 MB of PNG. The node tree for the same screen is 1–3 KB of JSON with labels, view ids and clickable flags — usually a few hundred tokens instead of tens of thousands.
Install
npm install dsh-phone-control --prefix ~/.dsh/profiles/desktop
Add "dsh-phone-control" to dsh.profile.bundles in that profile's package.json, then restart DSH.
The settings panel appears under Settings → Phone bridge.
Pair in five steps
- Install the Android app from the repository's releases page.
- Enable its Accessibility service.
- Open Settings → Phone bridge in DSH and read the 8-digit code — or scan the QR code the same panel renders (Scan pairing in the app fills in code, port and host).
- Type the code into the app and tap Connect.
- The panel turns green; the phone keeps a 30-day token and reconnects by itself afterwards.
Tools
| Tool | What it does |
|---|---|
phone_ui_dump |
Accessibility tree as JSON (maxDepth, maxNodes, includeInvisible) |
phone_tap_text |
Click the widget matching text / content-description / view-id |
phone_set_text |
Write into a field located by hint or view-id |
phone_screenshot |
PNG of the current screen |
phone_tap / phone_long_press / phone_swipe |
Coordinate input |
phone_input / phone_key |
Type text; send back / home / recents / notifications |
phone_open_app / phone_current_app / phone_list_apps |
Launch, identify, enumerate apps |
phone_fs_list / phone_fs_stat |
List a directory (default /sdcard) / stat a path |
phone_fs_read / phone_fs_write |
Read a file (UTF-8 or base64, paged) / write text or base64 |
phone_fs_mkdir / phone_fs_rm |
Create a directory tree / delete a file or directory |
phone_status / phone_pair_code / phone_disconnect |
Listener state, pairing code, drop session; forget: true revokes every issued token |
phone_notify |
Push a notification to the phone; probe: true reports its permission and switches |
Two things push themselves: a finished turn (" finished") and a question waiting for an answer
(" is waiting for you"), both labelled with the session title. The app has a Session notifications
card with the system permission plus a switch per category; phone_notify { probe: true } reads that state.
While the phone's foreground app is the bridge itself, the input tools answer SELF_PROTECTED; the app's
Permissions & security card has a switch (off by default) that lifts it. The file tools stay inside the app's
private directory until all-files access is granted (PERMISSION_DENIED otherwise).
Settings panel
Pairing code with countdown and a New code button, a live QR code of the pairing payload (GET /dsh-phone-control/qr
on the DSH web port), listening address/port (0 = auto), bind address, paired devices, open sessions, failed
attempts, and a Disconnect button. Port changes rebind immediately and persist to <profile>/phone-bridge.json.
App v0.0.38 also has a floating HUD: off by default, and when on it narrates the agent's actions in a
translucent strip at the top of the screen (actions plus a live thinking line streamed from agent/assistant-stream,
never the answer text; five lines, kept for the whole turn, then seven seconds of rest and a one-second fade). It hides
itself while the bridge app is in the foreground. Before a screenshot or node dump the overlay is detached and two
frames are awaited — one frame is not enough, the compositor still has the previous frame — so the agent never sees
its own HUD. Your manual screenshots do. The plugin pushes to it from sessions that called a phone_* tool within the
last twenty minutes, so a long reasoning stretch after a tap keeps updating instead of freezing.
Protocol
Binary frames (DSHP, 19-byte header, 8 MB cap). Handshake: HELLO → CHALLENGE → AUTH → OK, session key
from ephemeral ECDH (prime256v1) + PBKDF2-stretched pairing code + a 32-byte deviceKey the phone issues at
pairing time, all mixed through HKDF; HMAC proofs both ways, then AES-256-GCM with the sequence number in the
nonce. Only the SHA-256 of the pairing token and the matching deviceKey are stored — both sides must hold their
half or the handshake fails with BAD_PROOF (0.1.11+).
Node version
Requires Node.js 20 or newer. Runtime dependencies: ws, qrcode (the SVG QR route).
License
MIT