Inspecting Android's HTTP traffic with mitmproxy
The official Android documentation recommends using Android Studio’s profiler to inspect all network traffic, including HTTP requests.
However, such profiler only works with applications using either the
HttpURLConnection library or the
OkHttp library. This post explains how to
inspect HTTP requests made from an Android application regardless of which
libraries the application uses. This is achieved via an ad-hoc gateway running
Although parts of this post are specific to Android, the general principle can be used with any device that produces HTTP requests.
The basic idea is to build a proxy that will act as a gateway to the rest of the internet for the Android device. This proxy will intercept all HTTP traffic, including HTTPS traffic, and offer us a chance to inspect it on-the-fly.
In order to build our proxy we need a dedicated machine connected to the same network to which our Android device is connected. This needs to be a physical machine, preferably running Linux. An old computer, a SoC board - there is a lot of inexpensive hardware that can be used for this. As for the distribution, almost all flavours of Linux should work. For the sake of simplicity, this example is based on the latest LTS release of Ubuntu Linux.
Setting up the proxy
Assuming a working installation of Ubuntu Linux, setting up the proxy is just a matter of installing and configuring mitmproxy.
First, we update our machine:
$ sudo apt-get update $ sudo apt-get upgrade
Second, we download mitmproxy’s latest precompiled
binaries. Unpacking the
tar xzvf mitmproxy-x.y.z-linux.tar.gz produces the
mitmproxy binary executable, ready for use.
Third, we follow mitmproxy’s official docs to
configure our gateway’s network routing. We create the file
/etc/sysctl.d/mitmproxy.conf and enter the following lines to enable ip
forwarding and disable ICMP redirects:
net.ipv4.ip_forward=1 net.ipv6.conf.all.forwarding=1 net.ipv4.conf.all.send_redirects=0
Now we run the following commands to set the rules that will cause HTTP and HTTPS traffic to be redirected to mitmproxy’s port:
$ iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 $ iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8080 $ ip6tables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 $ ip6tables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8080
At this point, installing the
iptables-persistent module automatically
triggers a procedure that persists these rules across reboots (answer
Finally, we restart the server with
$ shutdown -r now.
Configuring the device
HTTPS traffic can’t be deciphered unless it is encrypted with a valid certificate. Mitmproxy provides an easy way to install a CA of its own on the Android device, which it then uses to trick the Android device into accepting spoof SSL certificates. To set it up, we start mitmproxy in normal mode:
$ mitmproxy --showhost
Then, in the Android device’s network settings, we modify the network’s
parameters by setting the
proxy option to
manual and entering the gateway’s
IP address and port
8080 in the respective input fields. Once done, navigating
http://mitm.it address shuold reveal a page with a few icons. Clicking
on the Android icon should trigger the download of a
.pem file. Tap on it to
bring up the certificate installer.
Once the certificate has been installed, reset the network
proxy parameter to
Configuring the application
Android manages two kinds of CA certificates: system certificates and user certificates. Although system certificates can be used by Android itself and all installed applications, recent versions of Android (starting from Nougat) require application manifests to explicitly state whether applications require access to user CA certificates such as the one installed in the previous step.
To grant our application such access, we add the following attribute to the
application element in the
Then, we create the
res/xml/network_security_config.xml file and add the
following lines to it:
<network-security-config> <base-config> <trust-anchors> <certificates src="system" /> <certificates src="user" /> </trust-anchors> </base-config> </network-security-config>
Once done, we can build the app and deploy it to our device. It took me a while to figure this out. Big thanks to @elye.project for the nice guide published on Medium.
Configuring the device’s network and inspecting traffic
We’re now ready to inspect some traffic! We restart
mitmproxy with the
$ mitmproxy --showhost --mode transparent --rawtcp
mitmproxy in its so-called
transparent mode. We can now change
the network configuration of the Android device by switching the IP address to
static (instead of
DHCP) and setting the network parameters by hand.
Everything needs to be configured according to what works for the local network
with the exception of the
gateway parameter, which we set to the gateway’s
local IP address.
Once done, we can fire up our application on the Android device and the
generated HTTP traffic should start showing up in