This is a record of my process trying to intercept SSL traffic on Android apps.

Obtaining APK files

Most tutorials would instruct you to download the APK file from third-party sites like APKpure and APKmirror, or install the APK first on your device then pulling it to the computer using adb pull. Here I use another way: manually downloading from Aurora Store. Aurora Store is an unofficial, FOSS client to Google’s Play Store. This gives some advantages:

• Third-party APK sites might not have latest version of the APK.
• Third-party APK sites might not have the APK variant (CPU architecture, screen resolution, locale, etc) that fits your device.
• Aurora Store allows you to easily spoof device models and regions.

Steps:

1. If you want to download apps only available in a specific region, connect to a VPN of that region.
2. Select “Anonymous" when using Aurora Store
3. Search for the app. The search result will contain apps available in that region.
4. Go to the app info page.
5. In the upper right 3-dot menu, select “Manual Download"
6. The text field will be filled with the latest version code. If you want to download an older version you can change the code here. Note: a) old versions are not always available, b) every app has their own scheme naming the code, c) the code corresponds to android:versionCode property in AndroidManifest.
7. Click “Download". This will download all split APKs (or Dynamic Delivery) that fits your device profile into Internal Storage/Aurora/.
8. You don’t have to install the app, because we’re going to modify it.

Using rootless Xposed

Unmodified version of the app failed to run (black screen) on VirtualXposed.

TaiChi even failed to install the app. Taichi will ask to uninstall the app outside Taichi’s container, then after uninstallation nothing follows.

Modifying the APK

Modifying APKs usually involves these steps:

• Decompile: apktool does a good job
• Modify:
  • Source code: edit smali, I don’t need it here
  • Resources: AndroidManifest.xml and other XML files, turns out there are quite some difficulty with this part.
• Rebuild package: apktool usually does a good job, but there seems to be some methods that an app developer can take to make apktool fail rebuilding.
• Signing: no problem

network_security_config

Since Android 7, user supplied CAs are not trusted in apps anymore. Most guides (like this one) will suggest you to modify networkSecurityConfig in AndroidManifest to make the app trust user supplied CAs.

Unfortunately when I do this for some apps, apktool will fail to rebuild the app. There are many strange errors, some can be solved by instructing apktool to use aapt2, but I encountered many cases where even aapt2 wouldn’t even help. So I had to seek other ways.

AndroidManifest have a complex binary format. It is usually quite easy to decode binary AndroidManifest into readable XML text, in the process the decoder (apktool) will flatten special “binary pointers to external resource files" into text paths and handle other special structures to text. But when encoding text AndroidManifest into binary, there are ambiguities to external references, causing the encoding to fail. (I don’t fully understand this part.) Fortunately, for most apps, there is another way: mark the app debuggable.

debuggable

For many apps, the networkSecurityConfig is defined like this:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <base-config cleartextTrafficPermitted="true">
        <trust-anchors>
            <certificates overridePins="true" src="system" />
        </trust-anchors>
    </base-config>
    <debug-overrides>
        <trust-anchors>
            <certificates src="user" />
        </trust-anchors>
    </debug-overrides>
</network-security-config>

This means that if the app is debuggable, it will accept user supplied CAs.

Modifying debuggable property in AndroidManifest.xml is a much smaller change than modifying networkSecurityConfig path. In the worse case I can just use a hex editor to flip a few bits. And since the change is minimal, apktool should be able to rebuild the app.

In order to modify debuggable property, I tried many tools:

• https://github.com/ytsutano/axmldec.git
• https://github.com/endlesscpp/axmleditor
• https://github.com/hzw1199/xml2axml

But in the end I had success with Ele7enxxh‘s AmBinaryEditor. (Documentation: http://ele7enxxh.com/AndroidManifest-Binary-Editor.html )

I also wrote some shell scripts to help handle decoding, building and signing split APKs: https://github.com/pellaeon/AddSecurityExceptionAndroid

Full process

Assumption: you already have all split APKs stored in a directory apk1/

1. Decode all APKs:

~/projects/AddSecurityExceptionAndroid/splitApktool.sh decode apk1/
# Decoded apks are put under apk1_tmp/

2 . Binary edit AndroidManifest.xml using AmBinaryEditor:

cd apk1_tmp/xxxx.apk_unpack/
~/projects/AmBinaryEditor/bin/Release/ameditor attr --modify application -d 1 -n debuggable -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1
mv AndroidManifest.xml1 AndroidManifest.xml

3 . Build the APKs

~/projects/AddSecurityExceptionAndroid/splitApktool.sh build apk1/ # It will build from apk1_tmp/

4 . Install the split APKs onto the device

~/projects/AddSecurityExceptionAndroid/adbinstallsplitapk.sh apk1_new/

5 . Check if the package is installed as debuggable:

$ adb shell
j3y17lte:/ $ for p in $(pm list packages | cut -d : -f 2); do (run-as $p id >/dev/null 2>&1 && echo $p); done
com.xxx # If it shows the package id, you have success

Notes

# Uninstall APK using pm. Sometimes a package will not remove completely when you use the GUI, causing the installation to fail.

adb shell pm uninstall <com.xxx.packageid>

Forking AmBinaryEditor

During testing I fixed a few quirks of AmBinaryEditor, they are documented in the readme. https://github.com/pellaeon/AmBinaryEditor

For apps that doesn’t have debuggable property already defined

In the aforementioned scenario, debuggable property already exists in AndroidManifest.xml. But if it does not already exist, we need to add an attribute using AmBinaryEditor.

# WON'T WORK: Use this command to add an debuggable attribute to the application tag
~/projects/AmBinaryEditor/bin/Release/ameditor attr --add application -d 1 -n debuggable -r 16842767 -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1

Note: when adding attributes, we need to specify the resource id using -r and a decimal number. Refer to the android source code for system global resource ids. Resource id for debuggable is 0x0101000f , so in decimal it is 16842767.

Unfortunately the debuggable attribute would not be accepted when modified in this way. (It would still install fine but not debuggable.)

To solve this problem, I inspect the APK using aapt:

$ aapt list -v -a apk2.apk
[...SNIP]
    E: application (line=106)
      A: android:theme(0x01010000)=@0x7f12001e
      A: android:label(0x01010001)=@0x7f110bbe
      A: android:icon(0x01010002)=@0x7f0e0000
      A: android:name(0x01010003)="[REDACTED]"
      A: android:persistent(0x0101000d)=(type 0x12)0x0
      A: android:launchMode(0x0101001d)=(type 0x10)0x3
      A: android:alwaysRetainTaskState(0x01010203)=(type 0x12)0xffffffff
      A: android:allowBackup(0x01010280)=(type 0x12)0x0
      A: android:largeHeap(0x0101035a)=(type 0x12)0xffffffff
      A: android:supportsRtl(0x010103af)=(type 0x12)0xffffffff
      A: android:resizeableActivity(0x010104f6)=(type 0x12)0x0
      A: android:networkSecurityConfig(0x01010527)=@0x7f150003
      A: android:roundIcon(0x0101052c)=@0x7f0e0000
      A: android:appComponentFactory(0x0101057a)="android.support.v4.app.CoreComponentFactory" (Raw: "android.support.v4.app.CoreComponentFactory")
      A: android:isSplitRequired(0x01010591)=(type 0x12)0xffffffff
      A: android:debuggable(0x0101000f)=(type 0x12)0x1

Compared with aapt output from using ameditor attr --modify on another APK with existing debuggable attribute:

$ aapt list -v -a apk1.apk | less
[...SNIP]
    E: application (line=111)
      A: android:theme(0x01010000)=@0x7f12002e
      A: android:label(0x01010001)=@0x7f110c19
      A: android:icon(0x01010002)=@0x7f0e0000
      A: android:name(0x01010003)="[REDACTED]"
      A: android:persistent(0x0101000d)=(type 0x12)0x0
      A: android:debuggable(0x0101000f)=(type 0x12)0x1
      A: android:launchMode(0x0101001d)=(type 0x10)0x3
      A: android:alwaysRetainTaskState(0x01010203)=(type 0x12)0xffffffff
      A: android:allowBackup(0x01010280)=(type 0x12)0x0
      A: android:largeHeap(0x0101035a)=(type 0x12)0xffffffff
      A: android:supportsRtl(0x010103af)=(type 0x12)0xffffffff
      A: android:resizeableActivity(0x010104f6)=(type 0x12)0x0
      A: android:networkSecurityConfig(0x01010527)=@0x7f150003
      A: android:roundIcon(0x0101052c)=@0x7f0e0000
      A: android:appComponentFactory(0x0101057a)="android.support.v4.app.CoreComponentFactory" (Raw: "android.support.v4.app.CoreComponentFactory")
      A: android:isSplitRequired(0x01010591)=(type 0x12)0xffffffff

One difference I spotted is that, in the previous one, the debuggable attribute is positioned last, and in the latter one, it is positioned in the middle. And in the latter one, attributes are sorted in their resource ID (for debuggable the resource id is 0x0101000f, see the android source code for all resource ids).

Next, looking at AmBinaryEditor’s source code, in function AddAttribute:

        while(1)
        {
            if (list->next == NULL)
            {
                break;
            }
            list = list->next;
        }
        list->next = attr;
        attr->prev = list;

It appears that attributes are stored in linked lists, and when adding a new attribute, it is added to the end of the list. This fits our observation from aapt output.

So, in order to make it work, I need to insert the debuggable attribute in correct position. I quickly modified the AmBinaryEditor source code with a hard-coded position index 3:

        for ( int i=0; i<=3; i++ )
        {
            if (list->next == NULL)
            {
                break;
            }
            list = list->next;
        }
        ATTRIBUTE *attr_orignext = list->next;
        list->next = attr;
        attr->prev = list;
        attr->next = attr_orignext;

Then try to insert the attribute and build the APK again:

$ ~/projects/AmBinaryEditor/bin/Release/ameditor attr --add application -d 1 -n 'debuggable' -r 16842767 -t 18 -v true -i AndroidManifest.xml -o AndroidManifest.xml1
$ mv AndroidManifest.xml1 AndroidManifest.xml
$ cd -
$ ~/projects/AddSecurityExceptionAndroid/splitApktool.sh build apk2/

This time the attribute is correctly inserted in the middle, and successfully parsed upon installation! Success!

References

• Writing a binary AndroidManifest parser https://github.com/lulululbj/android-reverse/blob/master/android/xml/parse_xml.md
• Structure of binary AndroidManifest file https://bbs.pediy.com/thread-194206.htm
• How Android packaging works: https://juejin.im/post/5cc7b5b86fb9a032297b0a29
• Structure of binary AndroidManifest.xml https://juejin.im/post/5c2253f6f265da616d54377b
• Androguard also has a decoder for AndroidManifest.xml https://androguard.readthedocs.io/en/latest/intro/axml.html
• About split APKs (or Dynamic Delivery): https://blog.csdn.net/ximsfei/article/details/50886134

Root

With root, everything is possible. But I didn’t need to go down this path. Well, I had to admit that intercepting traffic in unrooted environment is time spent digging unimportant hole. Eventually I still rooted the phone to intercept the traffic, that is, after all, my real goal.

Magisk only

Xposed doesn’t work on Android 8.1+ yet, so if you need to get this to work, use Magisk, it works on most versions of Android.

This Magisk module will copy the user CA store into system CA store: https://github.com/NVISO-BE/MagiskTrustUserCerts

As of Magisk 20.3 . the repo above doesn’t take any effect, seemingly because it’s using an older Magisk module template that is no longer supported. A pull request is opened to fix it but has not been accepted by the original author yet. In the mean time download the module from here https://github.com/giacomoferretti/MagiskTrustUserCerts to get the working version. (Follow the Installation section to generate installable Magisk zip file.)

Note: How did I discover that the module was not working?
Go to /data/adb/modules/trustusercerts , and I found that post-fs-data.sh , which does exist in the repo, missing. If it installed properly, it should have existed in that directory.

In the end, after I finally got it to work, I found that the particular app that I was looking at seemingly employs some custom SSL pinning, so the Magisk module only allowed me to intercept some of the HTTPS messages. So I had to move on to the next approach.

Objection and Frida

• Running android sslpinning disable from Objection shell – doesn’t work
• Tried some popular Frida scripts, doesn’t work either.

EdXposed

In the end, I got everything working with EdXposed and this module: https://github.com/Fuzion24/JustTrustMe

I was able to intercept all traffic from the app.

References

General walk-through articles

• https://warroom.securestate.com/android-7-intercepting-app-traffic/
• Magisk https://blog.nviso.be/2017/12/22/intercepting-https-traffic-from-apps-on-android-7-using-magisk-burp/
• Frida https://medium.com/@buff3r/root-detection-ssl-pinning-bypass-with-frida-framework-31769d31723a

Other tools

• https://github.com/1n3/reverseapk
• APK Studio https://vaibhavpandey.com/apkstudio/
• Inspeckage https://github.com/ac-pm/inspeckage

Deobfuscation

• https://calebfenton.github.io/2016/04/30/dalvik-virtual-execution-with-smalivm/
• https://github.com/calebfenton/simplify