android-use

Give your AI agent an Android phone. Keep your hands on the leash.

Your agent opens apps, reads the screen, taps, types, and drives Chrome on a real phone or an emulator, in about two tool calls per task. You decide what it may touch.

Install · Connect an agent · Control · Emulators · Documentation · Changelog

What is android-use?

Your agent lives in a terminal and a browser. Most of the apps people use every day live on a phone. android-use connects the two: it gives the agent you already use (Claude Code, Codex, Cursor, Gemini CLI, and others) hands on a real Android phone or emulator.

There are two pieces. A small helper app on the device reads the screen through Android's accessibility service and carries out actions. au, on your computer, talks to the helper over ADB and gives your agent two tools: one to look, one to act. The agent asks in plain commands like open app "Settings" then tap "Network & internet" and gets back a short answer: the screen it reached, and after later actions, only what changed.

• It reads text first. The screen comes back as a list of what is there: labels, buttons, switches with their state (sw Wi-Fi=on), and fields with their contents. Agents act on labels, not on guessed coordinates. When an app shows little text, a small screenshot with a numbered box per control comes along, so tap e4 still works.
• It checks itself. Every action is matched against the screen it was planned from, journaled so it runs at most once, and confirmed after. A locked phone, a screen that is off, or an ambiguous label stops it with the reason and the one step that fixes it, instead of a blind tap.
• It remembers what worked. Save a command that worked as a routine and run it again in one call. Every session leaves a trace you can review, export, or turn into a test.
• You stay in charge. A policy.toml decides which apps the agent may open, what it may capture, and how many steps it gets. Passwords are typed for the agent without the agent ever seeing them. For anything sensitive, the agent can ask the phone itself: you see the exact steps and tap Allow once or Deny, and the approval is void if the screen changed while you decided.

Developers use it to let their agent run the app it just built, report what it sees, and keep those runs as tests. Agents use it for real phone tasks end to end: checking settings, working through an app, reading a sign-in code from a notification, filling a form in Chrome. With a throwaway emulator, all of it can happen while your own phone stays in your pocket.

One call, then only what changed

A real session on an Android emulator, over MCP. The agent sends one command; the result is the screen it led to. android_act: open app "Settings" then tap "Network & internet" Done. Now on screen: Network & internet Navigate up Internet: AndroidWifi SIMs: T-Mobile sw Airplane mode=off Hotspot & tethering: Off ... android_act: tap "Airplane mode" Done. Changed: ~ Internet: Off ~ SIMs: T-Mobile [disabled] ~ sw Airplane mode=on The picture is what the agent gets when a screen has little text: 17 KB, with a numbered box per control.

Measured on an API 36 emulator over MCP, 13 tasks run 10 times each, against the same tasks before the changes in 1.1 (method and numbers):

	Before	Now
Tool calls per task	5.9	2.6 (2.3× fewer)
Input tokens per task	5,356	1,186 (−78%)
Fixed tokens per turn (instructions + tool definitions)	834	396 (−53%)
Output tokens per task	117	53
Tasks that succeeded	64%	98% (127 of 130)

Get started

npx -y android-use@1.4.0 setup

That is the whole install for most people (it needs Node.js 20.11+). setup puts the helper on your device, registers android-use with every agent it finds, and ends by reading your screen back to you. Or tell your agent to do it:

Run `npx -y android-use@1.4.0 setup`, follow its prompts, then use android_read and android_act.

You need Android 8 or newer with USB debugging on, or an emulator. No root. Android asks you to allow USB debugging and to turn on Android-Use under Settings → Accessibility; android-use never skips those prompts.

Homebrew, shell installer, Windows

brew install austinintelligence/tap/android-use && au setup
curl -fsSL https://raw.githubusercontent.com/austinintelligence/android-use/v1.4.0/install/install.sh | ANDROID_USE_VERSION=1.4.0 sh && au setup

On Windows, run install/install.ps1 in PowerShell, or unzip the archive from Releases and run .\au.exe setup. au runs on macOS (Apple Silicon and Intel), Linux (x86_64 and arm64), and Windows (x86_64 and arm64). Every download is checked against SHA256SUMS, and the archive contains au with the helper built in. Step by step →

To register one agent yourself: au install claude (also codex, cursor, gemini, vscode, claude-desktop, opencode, all), or au install --dry-run to see what would change. Only the android-use entry is touched (client details). Coding agents can start with AGENTS.md.

Two tools, plain language

android_read looks: screen, page, notifications, routines, policy, help. android_act does:

tap "Wi-Fi"                     set "Airplane mode" on           type "hello" in "Message"
open app "Chrome"               scroll down until "Privacy"      open app "Settings" then tap "Battery"
wait for notification from "Bank" up to 60 seconds
type secret "bank pin" in "PIN"
save routine "set timeout" from last with "30 seconds"
run routine "set timeout" with "1 minute"
ask phone "Pay $25 to Alex?" then tap "Pay $25"

The same commands work from a shell, no MCP needed: au read screen, au act 'tap "Wi-Fi"'. Miss a label and the reply lists the near matches and a ready-to-send fix. Hit a locked phone or a stopped helper and it says exactly that. Every form: android_read help, or the agent protocol.

What your agent can do with it

• Use apps the way a person would: open, read, tap, type, scroll, go back, chain steps with then
• Drive Chrome: open pages, click, fill, read, screenshot, through Chrome's own debugging socket
• Remember routines: save a command that worked, with a value as a parameter, and replay it in one call
• Sign in safely: type a stored secret into a password field without seeing it, and read a one-time code from a notification (read code from notification "Bank") for apps you list
• Read the phone: notifications (and act on them) and one-shot location, when you allow them
• Capture: screenshots; camera, microphone, and screen recording with Android's permission
• Turn runs into tests: au trace to-test writes a session out as a test, and au test runs files of the same plain commands with expect screen checks and JUnit output for CI (app testing)
• Work on emulators: managed emulators with snapshots, side by side with your real phone

Capability details →

No phone handy? Use an emulator

au emulator doctor           # needs the Android SDK; if it finds none, install Android Studio or the command-line tools
au emulator install          # fetch the missing emulator package and system image (you accept Google's license)
au emulator create demo
au emulator start demo
au --target demo setup       # then turn on Android-Use in the emulator's Accessibility settings

Each emulator is its own target with its own enrollment, journal, and serve lock, so an agent can work on one while you use your phone. Snapshots, restarts, and identity checks: Emulators and targets.

Safe to hand your phone to

• One device. A session is bound to one enrolled phone or emulator, checked on every connection, and the helper must answer a fresh cryptographic challenge before au sends it anything.
• Your policy, your rules. policy.toml sets allowed and denied apps, extra risky words, a step cap, whether anything may be captured, which apps codes may be read from, and the one folder files may move through. A file that does not parse stops the session; it never means "no policy". While a denied app is in front, only back and home work.
• Secrets stay secret. Values live in the system's secret store (macOS Keychain, Secret Service on Linux, per-user DPAPI on Windows), are typed only into a field the helper flags as a password field in an app you allowed, and are removed from every reply, error, trace, and routine.
• Approvals happen on the phone. While the prompt waits, the helper runs no actions at all, so no agent can tap Allow for you, and it never shows over the lock screen. The agent also cannot answer Android's own system permission prompts.
• You can stop it. au watch prints a link to a page on your computer that shows each command as it runs, with Pause and Stop. The helper's notification has a Stop button too.
• Nothing extra. The helper has no internet permission, au exposes no shell, and every action is bounded in size and time.

The full model: SECURITY.md and the control guide. Under the hood: Architecture.

Documentation

I want to…	Read
Get it running	Install, Troubleshooting
Connect Codex, Claude Code, Cursor, …	Agent quickstart, client setup
Set limits, secrets, approvals, routines	Control
Test my app	App testing
Look up a command	CLI, agent protocol, environment
Run on emulators	Emulators and targets
Understand the cost numbers	Token budget
Run it unattended	Production guidance, security model
Contribute or cut a release	Development, testing, releasing

FAQ

Does it include an AI model?

No. It gives the agent you already use a compact, checked Android interface.

Can it type my password without learning it?

Yes. Add it once with printf %s "$PIN" | au secret add "bank pin" --apps com.example.bank. The agent then sends type secret "bank pin" in "PIN", and the value never appears in anything it gets back. au secret list shows names only; au secret remove NAME deletes one.

Does it need screenshots?

Usually not. The text screen is smaller and names every control the agent can use. When a screen has little text, or a label is not found, a compressed screenshot with numbered boxes is attached, and tap point X Y on it maps back to exact device pixels.

Does my phone need to be rooted?

No. It uses USB debugging and Android's accessibility service, both granted by you.

Can it wander onto the wrong device?

No. A session is bound to one enrolled device or emulator target, checked every time it connects.

Wi-Fi instead of USB?

Release testing uses USB. Wireless ADB works as an advanced transport; secure it like physical access.

Where do captures, traces, and logs go?

They stay on your machine in a private folder. Captures and traces older than 7 days are deleted the next time Android Use opens that device, and the journal keeps operation metadata, never screen or media contents. au uninstall removes the helper, the journal, and captures; stored secrets stay in your system's secret store until au secret remove. See Security.

Status

android-use targets Android 8 and newer; the automated checks run on an Android 16 (API 36) emulator. Optional capabilities vary by device, and au doctor says what a connected one supports. The 1.4 control features are covered by unit and flow tests; the changelog lists what has not been run on a device yet.

MIT licensed. Contributions are welcome: see CONTRIBUTING.md. If android-use earns a place in your setup, a star helps the next person find it.