Horizontally scaling a Rails app at the push of a button is something that I have never quite managed to get right. Using chef to provision new app servers, reconfigure a load balancer and then deploying code using Capistrano is about as close as I’ve managed to get it. Until now.
Recently at work there’s been a debate about how to automate infrastructure management, and to make deploying multiple resilient and load balanced applications easy. The decision came down to either Docker Swarm or Kubernetes, so I decided to see how things might look using Swarm. I’m a big fan of Rails and developing applications for it takes up the majority of my coding time, so deploying it was the obvious way to put Swarm to the test. I decided to use Docker Compose for orchestration because it seemed to be the easiest way of going about it.
Setting up the cluster
After reading through the docs for Swarm and getting my head around what a Swarm cluster consists of, I followed this guide and made myself a scalable (but highly insecure) Swarm cluster. Here’s a quick diagram of what my cluster looked like:
Whether or not this is an optimal setup is beyond the scope of what I’m doing here. The idea behind my layout is that all data stores (such as Postgres, MySql etc) will be constrained the storage node, meaning we don’t have to worry about using network storage drivers for volumes. All application containers will run on any of the worker nodes which can be added and removed as needed, and then the frontend node is running Interlock v1.0.0 along with HAProxy exposed on port 80.
Configuring the Rails application
With the swarm cluster up and running the next job is to configure our Rails app in preparation for deploying to the Swarm. The idea here to to make the app completely isolated, so you can run any number of them without worrying about them bumping into each other, and having no external dependencies which can prevent the container from starting up. The 12 factor app methodology is key here and if you follow all of the steps on there you should be ready to go in no time. Specific to Rails, there are just a few things you need to take into account.
- All database config should come in the form of environment variables
- All environment specific config should come in the form of environment variables
- Static assets should be served without the need for a reverse proxy
- Logging should all be done to STDOUT so Docker can control where it goes
The rails_12factor gem takes care of two of these concerns for you, so once you’ve sorted this the last thing to do is create your Dockerfile. This will be a very minimal file, as all it has to do is install your gems, precompile assets and kick off your web server. Here’s one I made earlier:
Putting it all together
The final job is to put it all together in a docker-compose.yml file ready for deployment into your cluster. This will have to include a few extra things from a standard compose file:
- Your app has to be constrained to worker nodes
- Your database has to be constrained to the storage node
- You have to specify a hostname which can be resolved the the frontend for your app containers (for interlock to work)
Here’s my compose file as an example:
Now you can run
docker-compose up against your cluster and it should bring up your application. Success. If you want
to view the app running then you’ll need to add an entry to your hosts file (/private/etc/hosts on OSX) pointing to
your frontend node from
rails-demo.local. Obviously in production you’ll have real DNS names and pubic IPs so this
won’t be necessary.
If you’ve deployed a Rails app before you’ll probably be wondering why I’m letting Rails serve its own static files instead of using a reverse proxy such as Nginx which does a far better job of it. Well there’s one more piece to this setup which I haven’t mentioned yet, which is a caching server for your assets. Usually in production you’ll use an external paid service for this as they will do a good job of it with minimal effort. However if this is not an option you can set it up yourself using Nginx.
The way this works is that you tell Rails to get your assets from the remote Nginx server. This server is configured to work as a reveres proxy to your app, so it will send the request for an asset back to your app which will serve the static asset to Nginx, which then sends it back to the client. The first time around this is a pretty bad way of doing this, but combined with Nginx’s caching config this becomes extremely efficient when anybody else asks for the asset. using a few lines of config, you can get Nginx to keep hold of the asset it previously served ready for the next client who comes asking. Here’s example Nginx config which turns on this functionality.
The important thing with this config is that the
Host header is set to the hostname of the upstream, otherwise
Interlock won’t be able to figure out where to send the request. There’s a whole lot of configuration you can use
with Nginx’s caching, and what you see above is just taken from the example on their website.
Add this config into the Nginx image and push it to your repo ready to deploy it. You can add it to your compose file by adding the following block to your services:
Note that the
rais-demo.local address had to be added manually because otherwise Nginx will have no idea where
to find the upstream. Again, you’ll have to add the hostname
assets.rails-demo.local to your hosts file to try this
Now that nginx is ready we need to tell Rails to use our new static files cache to serve its files. Add this line to
config/environments/production.rb to do this:
And there you go. Updating your images using
docker-compose pull followed by
docker-compose up against the Swarm
should spin up your applicaiton, cache and database ready for use.
So the whole point of this was to get to a point where the application can be easily scaled horizontally. This is where
scale function comes into play. It allows you to spin up (or down) any number of each service defined
docker-compose.yml file in an instant. So if your Rails app is starting to choke under the load, just run
docker-compose scale rails=3 and 2 more will appear spread as evenly as possible across your worker nodes. Then when
load isn’t as heavy and you want to free up some resources,
docker-compose scale rails=1 will cleanly tear down the
Interlock will handle the load balancing so you don’t have to worry about that, and Nginx should be fine as a single node since all it’s doing is serving cached files (which the client’s browser can also cache). Nginx can be scaled in the same way as Rails was if need be though.
So there you go. A rails app scaled horiontally across multiple nodes with cached assets as well. There are a few tools to get to grips with but nothing too complex, and once everything is up and running your app will be both as resilient and as performant as you need.