departureboard.io: Now powered by the departureboard.io API

I launched departureboard.io back in late December 2018. At the time the departureboard.io API didn’t exist, so all queries for rail data went directly to the National Rail OpenLDBWS SOAP API. Developing departureboard.io was painful, because interacting with SOAP is simply not fun. The whole reason I developed the departureboard.io API in the first place was so other developers wouldn’t experience the same frustrations and challenges that I did. However, after releasing the departureboard.

Latest departureboard.io API improvements: Intelligent Caching

Over the past few weeks, I’ve spent some considerable time making improvements and optimisations to the departureboard.io API. This is a never-ending journey. I have a full roadmap of improvements and features that I want to implement over time. But for this blog post, I want to focus on one improvement in particular: Intelligent Caching. The Opportunity As I have described in previous posts, the departureboard.io API provides a RESTful interface on top of the official National Rail SOAP API.

Exporting iOS 13 app usage to Prometheus, and graphing with Grafana

I spend a lot of time on social media apps on my iPhone. Reddit, Instagram and Twitter are the 3 main culprits. I’m not an active contributor on any of the platforms, but I do find myself frequently checking the apps and getting lost in that infinite scroll. During the day I am pretty busy with work and don’t really check them, but in the evenings and weekends when I’m not in front of my laptop or out with friends, I am guilty of spending a lot of time on them.

K8s: Stopping kube-system pods from preventing cluster scale-down

I’m now at the point where I run a fair few production services from Kubernetes, the most successful being departureboard.io and the departureboard.io API. To make sure I can meet the fluctuating demand for these services, in a cost-efficient fashion, I take advantage of the Kubernetes Cluster Autoscaler. I won’t go into too much detail about how this works, but in short, it enables my cluster to automatically scale up if the total resource requests from my application pods exceed the available capacity in the cluster.

Latest Project: departureboard.io REST API

Wow! It has been over 6 months since I last added a post to my blog. Definitely doesn’t feel that long. Since my last post I’ve been pretty busy, both at work and in my own time. Work aside, I’ve been moving more and more towards software development. I’ve always been largely infrastructure focused but there is only so much fun you can have without some actual software to build infrastructure around.

Kubernetes Cronjob: Automated MySQL database backups

Update: This project has been renamed to kubernetes-s3-mysql-backup, which better describes its function. The remainder of the post remains accurate. As mentioned in my last post about my Kubernetes Architecture. I recently started investigating how I could perform regular backups of the MySQL database that sits behind this blog. The goal was to have an automated backup taken at least once every 24 hours, saved to a durable location external to my current hosting provider.

This blog now runs on Kubernetes: Here's my architecture

As I alluded too in my previous post, I’ve recently been working in moving my blog (and all of my services) to Kubernetes. Up until this point, I’ve been running all of my services in containers, but on specific machines with Docker installed. I had a pretty sophisticated and automated deployment process but it was only ever an interim solution before I delved into Kubernetes. The decision to move to Kubernetes was further expedited by being given access to the Digital Ocean Kubernetes Limited Availability Preview.

Kubernetes & ingress-nginx: Performing HTTP Redirects

I’ve recently been doing a lot of work with Kubernetes (in fact, this whole blog now runs on Kubernetes, with more posts to follow). To avoid lots of load balancer charges, I’ve been using ingress-nginx. This allows me to have one load balancer for all of my web traffic (pointing to the ingress-nginx service) and offload traffic to the appropriate backend based on ingress definitions. The below diagram shows how this works.

Automating my Cloud Infrastructure: Terraform

Having had a very busy few months, changing jobs and moving to a new house, I’ve finally found the time to write a new post! In what little free time I’ve managed to grab over the last few months, I’ve been working on completely overhauling my infrastructure. I recently started becoming increasingly frustrated with how long I was spending on deploying and configuring the servers that I use for some of the applications I host.