July 27, 2019 · Projects ios-upe

Exporting iOS 13 app usage to Prometheus, and graphing with Grafana

I spend a lot of time on social media apps on my iPhone. Reddit, Instagram and Twitter are the 3 main culprits. I'm not an active contributor on any of the platforms, but I do find myself frequently checking the apps and getting lost in that infinite scroll.

During the day I am pretty busy with work and don't really check them, but in the evenings and weekends when I'm not in front of my laptop or out with friends, I am guilty of spending a lot of time on them. I've been on a mission to cut down on my usage for a long time, particularly for the Facebook owned services. I deleted my Facebook last year but have found it difficult to get away from the other services. This is made especially difficult by the fact I have friends all over the place that I keep in touch with via these apps.

Screen Time on iOS has been a useful addition to monitor just how much I am using these applications, however after upgrading to the iOS 13 Developer Beta a few weeks ago, fuelled by a reasonable level of boredom, the introduction of the new Shortcuts application gave me an idea!

I currently make heavy us of Prometheus and Grafana for monitoring my infrastructure and applications, anything from how many nodes are currently in my Kubernetes Cluster, through to how many Departure Boards I have generated on https://departureboard.io. Prometheus scrapes and collects the metrics, and Grafana enables me to visualise them. An example of the dashboard for of my clusters is above.

Here is where I had my idea, what if I could export metrics on how often I use certain applications on my iPhone, and then visualise them in Grafana?

Hello iOS 13 Shortcuts...!

Shortcuts is an application by Apple introduced in iOS 12. In its initial iOS 12 incarnation it was designed to automate certain actions based on user-defined Siri commands. Which wasn't too useful for what I was looking to do.

However, iOS 13 contains a complete overhaul of the Shortcuts app. Alongside now coming as a default application in iOS, it contains a whole plethora of new features. One of which is the "Automation" functionality.

The Automation functionality allows you to create workflows that trigger automatically based on certain actions performed, such as:

Actions can then be performed on the back of these triggers, such as:

Aha! I had discovered the two required features to make my idea a reality. The "When a certain application is opened" trigger, and the "Make a HTTP request" action. This gives me (almost) everything I needed to ingest data into Prometheus.

A little bit on Prometheus metric collection

I am not going to go low level into how Prometheus works in this post (see the Prometheus Overview for that), but before I go into how I approached the task of exporting iOS app usage into Prometheus, it is important to understand how Prometheus ingests metrics.

Prometheus server works on the concept of "scraping" metrics. You can't send metrics directly into Prometheus (for example via an API call). Instead Prometheus periodically scrapes a /metrics HTTP endpoint on specified targets, and ingests the metrics from there.

This works perfectly for targets that are reasonably long-lived, such as a web or application server, but it presents challenges for other short-lived services such as Cronjobs (or an iPhone that doesn't have any means of exposing a /metrics endpoint).

Prometheus Pushgateway

Prometheus have a solution to solve for this problem, and it's called the Prometheus Pushgateway. Pushgateway allows short lived jobs to push metrics into the Pushgateway, so that the jobs can complete without having to wait for Prometheus to come along and scrape its metrics.

These metrics are sent to the Pushgateway using HTTP POST requests, via an interface described by Prometheus as "vaguely REST-like". The Pushgateway then exposes these metrics itself (effectively acting as a metrics proxy), via the /metrics endpoint, and Prometheus comes along and scrapes them.

With this solution, it looked like I had a way to push my iOS app usage into Prometheus. So at this point I could simply crack on and create the automation's to send my app usage into Pushgateway.

Sadly not!

Unfortunately, the Pushgateway solution has two limitations that can't be ignored when looking to export metrics from iOS into Prometheus:

ios-usage-prometheus-exporter

I'd already become quite invested in the idea of exporting my metrics into Prometheus, so didn't want to fall at this hurdle. So I fired up Visual Studio Code and wrote an API in Golang that performs similar functionality to Pushgateway, with a couple of key differences:

The above diagram shows the high-level architecture for ios-usage-prometheus-exporter. You can grab the code and documentation on GitHub.

The Results

Finally, I was in a position to start ingesting some metrics into Prometheus. I decided to configure an automation for my most frequently used applications: Mail, Instagram, Reddit, WhatsApp, Messages, Safari, Spotify, Monzo and Photos. This extended the scope beyond social media, but I was interested to see my usage as a whole.

After letting it run for just over a week, the numbers were pretty interesting. There are loads of ways you can cut the data, but as an example, to see my most used applications over the past 7 days I ran the following query in Prometheus.

sort_desc(sum(round(increase(ios_app_open_total{deviceName="BensiPhone"}[7d]))) by (appName))

This returned some interesting numbers, Mail was my opened application (at 513 opens in 7 days), primarily due to checking work emails. Instagram was a reasonably close second at 357 opens in a 7 day period. This averaged 51 opens a day, which is pretty scary!

I'm still in the process of playing with my Grafana dashboard for these metrics, and so far haven't had the time to perfect it. I did however quickly create a table and a bar graph to visualise the metrics.

Mission Success!

And that's it, mission accomplished. It ended up being a little bit more involved than I'd first anticipated. I planned on putting something together really quickly using Prometheus Pushgateway, but it was a valuable learning exercise working around the limitations by creating my own exporter. Ironically, the distraction of setting this up slightly reduced my social media usage, but I'm determined to cut it down even further.

I'm going to continue playing with the Shortcuts app and see what other useful information can be exported from iOS, I'll post updates to this blog if I come up with anything interesting.

As mentioned above, if  you want to set this up, the code for ios-usage-prometheus-exporter is available on GitHub alongside more detailed documentation.

  • LinkedIn
  • Tumblr
  • Reddit
  • Google+
  • Pinterest
  • Pocket