Case study: Shelly — an Android terminal and AI agent runtime built by directing AI

In short. I’m RYO ITABASHI, a Creative Director. Over eight months I directed AI coding agents to build Shelly, an open-source AI terminal and agent runtime for Android. I decided what to build and what to cut, and nothing counted as done until it worked on a real phone.

Period	Commits	Releases	Test devices
2026-03-04 → 2026-11-04	2,830	21 versioned	1 (Galaxy Z Fold6, N=1)

Shelly on GitHub · Demo videos (Shelly README)

Evidence 1

The problem

Phones are always on, always with you, and already hold your files, accounts and context. Yet running development tools or an unattended AI agent on one usually means installing Termux and setting things up by hand, or running a server somewhere else. The question was whether one Android app could ship bash, Node.js, Python and git, a terminal to use them in, and agents that run on a schedule. To run, it would need no PC, no Termux and no cloud runner. That is a requirement on the app at runtime. It says nothing about how the app was developed.

Evidence 2

The constraints

• One device. Everything was designed around a single Samsung Galaxy Z Fold6 and verified on it. Results are marked N=1.
• Android security as shipped. Shelly runs as an ordinary app under Samsung Knox’s SELinux policy. No root, no proot.
• AI writes the implementation. I don’t hand-write production code. Implementation is delegated to AI coding agents under my direction. In git, about 2,200 of 2,830 commits carry an AI author or co-author marker. The rest carry no marker either way, so git can’t prove the origin of every line.
• Third-party terms. Some routes that worked technically were ruled out because of vendor terms of service.

Evidence 3

Who decides, who implements, who verifies

• Decides — me. What Shelly is for, what goes on screen and what doesn’t, which features to cut, which risks are acceptable, and when something counts as done.
• Implements — AI coding agents (Claude Code, Codex). Kotlin and C native modules, the TypeScript app, the bash runtime, tests, investigation and first-pass reviews. At several high-risk points, Claude Code and Codex reviewed each other’s work, and I decided which findings mattered.
• Verifies — the Fold6. The Fold6 was the primary device for product decisions and the only hardware-verification target. On-device checks were often run from a PC over adb by an AI agent. Whether a result counted as a pass was my call.

The project doesn’t depend on anyone’s memory. docs/superpowers/DEFERRED.md is the single source of truth for deferred, descoped and known-limitation work, and every entry carries a reason and a priority. CLAUDE.md contains the rule: no verbal “let’s do that later”; every deferral goes into the file. Product calls are recorded in my own words. One feature, for example, was removed after I said 「無くしちゃおうぜ」 (“let’s just get rid of it”).

Evidence 4

Three turning points

1. Knox blocks shebang scripts and am start

The early design put small #! shell scripts on PATH and used am start to pass URLs from the shell to the app. On the Fold6, neither worked. The kernel’s script loader needs read access that Knox’s policy denies on the app’s own data files. Every interpreter line failed the same way: #!/system/bin/sh, a bash in $HOME, linker64. am start from the app’s uid was rejected by the activity manager. The fix was structural. Every command visible on PATH became a native binary shipped in the APK and symlinked from $HOME/bin. Messages from the shell to the app became a file queue that the app drains every 250 ms. Running ./script.sh directly still fails on this device, and the docs say so: use bash script.sh.

Evidence 5

2. app.act: shipped as an experiment, removed two days later

app.act let an agent operate other apps, such as sending LINE messages or posting on X, through Android’s Accessibility Service and recorded recipes. The experimental Phase 1 went live on 2026-08-29. A single on-device test session found three separate structural bugs: a timing miss on cold start, a matcher that could bind to several screen elements at once, and a focus-stealing bug in the recipe-capture UI that couldn’t be fixed. Even limited to two apps, automation that reads the screen broke whenever the target app changed. On 2026-08-31 the whole feature was removed: the Kotlin service, the TypeScript, tests, docs and about 25 i18n keys. Posting moved to API connectors. Cross-app tasks later came back through a separate app (ClosePaw), with approval on the phone.

Evidence 6

3. A gate bypass found by testing the gate

The highest permission level, “run fully without approval”, auto-approves file operations without asking. A user complaint led to a fix: a quoted \| in a grep pattern was being read as a pipe. During that work, adversarial inputs were sent straight to the existing checker on main, and several were wrongly approved. In one, a quoted "<<EOF" was taken for a heredoc, so a later cp evil ~/bin/git was skipped as heredoc text. In another, .. was hidden inside a brace expansion. In a third, a cd that might fail was trusted as the new working directory. On 2026-11-02 the checker was replaced with a lexer that reads quotes, escapes, comments and heredocs the way the shell does. The next day the gate also learned to catch network tools disguised with quotes, such as cu''rl, "c"url and $'\x63'url, and that fix was verified on the device (build 2821). The lexer change itself has passed its tests but not yet been checked on the device.

Evidence 7

Other calls, briefly

• Narrowing the CLI promise. v5.3.1 (2026-05-14) officially supported Claude Code and Codex in the terminal. The on-device Claude Code path then needed a run of fixes in mid-May without becoming stable, and on 2026-05-29 the Claude Code and Gemini CLI integrations were removed. From then until late October 2026, the terminal CLI was Codex, with other providers used as explicit API backends; Claude Code returned as a manually staged official binary on 2026-10-31.
• Freezing a route that worked. On 2026-08-31 a proof of concept unpacked the Bun single-executable that Claude Code ships as and ran its JavaScript on Node. --version worked. The coding tool’s safety classifier kept blocking this class of operation. Looking into why, I judged that the route likely conflicted with the vendor’s terms, and froze it. That was the project’s risk judgement, not a legal finding. Claude Code later returned through Anthropic’s official linux-arm64 glibc build, run unmodified. The lesson: check vendor terms before touching a proprietary binary.
• A 27B local model, rejected. It loaded on the phone but was too slow to hold a conversation with (2026-07-15).
• Plain chat instead of a confirm card. I tried structured cards for confirming agent registration several times, and chose plain chat text instead.

Evidence 8

The security boundary: the approval gate

If AI writes the code and agents run unattended, the boundary can’t rest on trust. It has to be built into the structure. In Shelly, every command an agent’s CLI worker (Codex or Claude Code) proposes goes through one driver (shelly-agent-driver.js) and one gate. The gate checks command safety level, workspace boundaries and a policy layer. Some requests, such as critical commands and writes to the policy itself, are refused before a person ever sees them.

• Unattended runs decline anything that would need a human.
• Attended runs escalate to the phone as a notification. The approval comes back as a reply signed with a key held in Android Keystore.
• In the ChatGPT → phone-app flow (ClosePaw), ChatGPT can ask for a phone action but can never approve one. The approval comes only from the phone. The experimental Codex remote-control host (shelly codex pair) is a separate path, and this statement doesn’t cover it.
• “Run without approval” is off by default and set per agent. The “standard” level only covers soft cases. The “all” level applies only to attended runs, and only to agents that don’t read untrusted input such as web research, URLs or notifications. It auto-allows file operations (create, edit, move, delete) inside the app’s home directory, excluding dot-directories and runtime folders such as ~/bin and ~/models. It also covers /tmp and $TMPDIR, and read-only syntax checks (node --check, python3 -m py_compile, bash -n) on files in the workspace. Running scripts or programs, network commands, anything touching secrets, link-creating commands and system commands still ask. The flag is stored in Keystore-backed storage, not in the agent’s own file, which a run could rewrite.
• Open question. It is not yet confirmed whether tool calls Codex makes through MCP servers or ChatGPT apps bypass the gate. On 2026-11-04 an on-device test ran a search through an installed ChatGPT app with no approval, and a code review points the same way. It is tracked as P1 and is not yet fixed.

Evidence 9

How verification works

There is one test device, a Galaxy Z Fold6 (N=1). A feature isn’t called done until it has been seen working on that phone. The README labels results “On-device verified (Galaxy Z Fold6, N=1)”, and DEFERRED.md keeps a separate “implemented, not verified on device” state, which many entries still carry. Device checks run over adb, often driven from a PC by an AI agent: UI automation, logcat filtered by Shelly’s own log tags, and alarm state from dumpsys. The agent pipeline’s regression tests run the real generated scripts as real processes instead of hand-copied re-implementations. Risky changes, such as native code, OAuth, IPC and security boundaries, get a review from a second agent before they land. Users on other phones are invited to submit shelly doctor --report.

Evidence 10

Results (as of 2026-11-04)

• 2,830 commits on main between 2026-03-04 and 2026-11-04.
• 21 versioned releases, from v4.2.0 (2026-04-01) to v8.2.0 (2026-11-03).
• 68 stars and 12 forks on GitHub.
• The full unattended cycle (register in plain language, fire on a schedule with the screen off, report back) verified end-to-end on the Fold6, N=1.

For scale: the codebase produced under this AI-directed workflow is about 225,000 lines of app code (physical lines including blanks and comments; tests, generated files and the adapted Termux code excluded), plus about 101,000 lines of tests in 443 files, with roughly 6,150 test cases (a grep count, not a pass count). These figures show size only. They are not a quality metric, and they don’t prove who wrote each line.

Evidence 11

Limits that remain

• One device. Nothing has been verified on another phone or on another manufacturer’s Android build.
• Fixed, not yet checked on the device. The quote-aware lexer for the “all” level, and four fixes for bugs that came up in on-device agent runs (result order after “OK”, a Codex step silently falling back to the local model, the MONITOR label, a skill-save prompt after a skipped step). Both have passed their tests but not been checked on the phone.
• The MCP question. Whether Codex’s MCP / ChatGPT-app tool calls bypass the gate is still open (see above).
• The checks are lexical. The gate reads the command text. It doesn’t follow what a file operation is later used for. That is documented as an accepted constraint.
• Self-verification. The tests, reviews and device checks are all run by the project itself. There has been no outside security audit.
• Not the official Codex. Codex runs as an open-source Termux fork, not OpenAI’s official binary.
• Distribution. Shelly ships through GitHub Releases only (no Play Store or F-Droid yet), and the APK is about 700 MB.
• Not phone-only development. The AI agents ran on the phone and on a Windows PC. The phone’s role was the product decisions and the hardware verification.

Evidence 12

What I’d do next

Close the MCP question first: give Codex workers no MCP servers or apps, or only a read-only allowlist, and run MCP tool calls through their own gate. Then get the “all” level and the S4 fixes checked on the device, and get results from a second phone and an outside security review.

Evidence 13

For talks or project inquiries, use the inquiry form.