This is a hobbyhorse fork for my silly idea to try putting a Model Context Protocol server inside of TalkBack, so we can see if a model can navigate an app with just the text Talkback would read and a few human gestures.

Talkback stuff:

This repository contains source code for Google's TalkBack, which is a screen reader for blind and visually-impaired users of Android. For usage instructions, see TalkBack User Guide.

To build TalkBack, run ./build.sh, which will produce an apk file.

Install the apk onto your Android device in the usual manner using adb.

With the apk now installed on the device, the TalkBack service should now be present under Settings -> Accessibility, and will be off by default. To turn it on, toggle the switch preference to the on position.