Healthchecks based loadbalance and failover with DNS using PowerDNS

Nowadays many enterprise DNS providers provide healthchecks based DNS failover solution, including from AWS Route 53. The premise of this is directing the traffic towards the available nodes only using the DNS authority chain for the records.

At Hopbox, we manage a bunch of our domains in-house using BIND9 and PowerDNS. We wanted to make sure the application is actually listening and available on a server before directing the traffic to it. In case one of the servers is not available or service is down, its IP address is removed from the DNS responses. We have chosen PowerDNS Lua Records for this purpose.

Lua records are Lua statements in PowerDNS, which need to be enabled using the [enable-lua-records] 1 flag in config. Lua records are quite simple to define:

app IN LUA A ( "ifportup(9001, {'192.0.2.1', '198.51.100.39', '203.0.113.126'," "{selector='all'}) ")

In the above statement, an A record for the subdomain app is defined, which monitors reachability on port 9001 for three defined nodes, i.e., 192.0.2.1, 198.51.100.39 and 203.0.113.126 and returns all up nodes in response (defined by selector=’all’). 2

That’s a mouthful of explanation for sure. In short, for a DNS query for the subdomain app, PowerDNS would return all available nodes (based on port 9001 availability).

selector can be changed based on geography as well:

app IN LUA A ( "ifportup(9001, {'192.0.2.1', '198.51.100.39', '203.0.113.126'," "{selector='pickclosest'}) ")

This modifies query response to direct traffic to the closest node on the basis of the requestor's geographic distance. Monitoring or availability checks are done in an async manner in the background, and we have observed check frequency varying from 2 to 5 seconds. The default TTL for response seems to be 120 seconds.

A full list of Lua functions available in PowerDNS can be found here. Most can be combined to do fine-grained load balancing.

From Lua Records documentation:

This is a PowerDNS specific feature, and is not (yet) standardized by the IETF or other standards bodies. We are committed however to interoperability, and strive to turn this functionality into a broadly supported standard.

So AXFR is only supported to other PowerDNS secondaries only.

A diagram showing Managed DNS service using this setup

At Hopbox, we also provide Managed DNS services, where you can point a CNAME for your application subdomain towards us, and we manage the healthchecks (port specific too) based on DNS responses (with our diverse, highly available DNS infrastructure). You may Contact us to enquire abut how we can help optimise your application delivery.

Further readings

PowerDNS Lua Records announcement blogLua Records documentation