How to Automate Certbot Renewal with HAProxy

So this is specifically for HAProxy on FreeBSD, but it should apply to other *nix systems as well. Basically, I use HAProxy as a reverse proxy to a bunch of servers I administer. I use Let's Encrypt for a certificate and I used certbot to generate that certificate. Generating the certificate for the first time is easy and has lots of documentation, but it wasn't initially clear on how I could easily set up auto-renewal. Here's how I did it.

If you've already set up TLS termination with HAProxy and certbot, you know you need to combine your Let's Encrypt fullchain and private key to get a combined certificate that HAProxy can use. You can do this by cat-ing the chain and key together like so:


cat /usr/local/etc/letsencrypt/live/$SITE/fullchain.pem /usr/local/etc/letsencrypt/live/$SITE/privkey.pem > /usr/local/etc/ssl/haproxy.pem

	  

In this example, $SITE is your domain name that you fed HAProxy when you created the certificate and haproxy.pem is wherever you're storing HAProxy's combined certificate. Your HAProxy config then points to that certificate like this:


macon% grep crt /usr/local/etc/haproxy.conf
        bind *:443 ssl crt /usr/local/etc/ssl/haproxy.pem

	  

And that was the end of the first-time setup. Then a few months later you probably had to do it again because Let's Encrypt certs are only good for 90 days in between renewals. To renew the certificate, you usually run certbot renew, it detects which certificates are present, and uses either the webroot or standlone server renewal process. Then you have to cat the fullchain and privkey together and restart HAProxy so it starts using the new certificate.

To automate those steps, newer versions of certbot will run any post renewal hooks (read: scripts) that you want. You can also configure HAProxy and certbot to perform the ACME challenge dance for renewal so that you don't have to use it interactively.

First, if you haven't already done it, change your HAProxy config so there's a frontend+backend for responding to ACME challenges. In a frontend listening for requests, create an access control list for any request looking for /.well-known/acme-challenge/. Send those requests to a backend server with an unused local port.


frontend http-in
		acl letsencrypt-acl path_beg /.well-known/acme-challenge/
        use_backend letsencrypt-backend if letsencrypt-acl
		...
backend letsencrypt-backend
		server letsencrypt 127.0.0.1:54321

	  

What this will do is allow certbot and Let's Encrypt to renew your server in standalone mode via your reverse proxy. As an added bonus it prevents you from having to open up an additional port on your firewall.

Now you've gotta configure certbot to do just that. A config file was created in certbot's renew directory for your site. All you need to do in that file is add a line to the [renewalparams] section specifying the port you're using in your HAProxy config.


macon% echo "http01_port = 54321" >> /usr/local/etc/letsencrypt/renewal/$SITE.conf

	  

Now you need the post-renewal hooks. I dropped two separate scripts into the renewal-hooks directory: one does the job of combining the certificate chain and private key and the other just restarts HAProxy.


macon% cat /usr/local/etc/letsencrypt/renewal-hooks/post/001-catcerts.sh
#!/bin/sh

SITE=(your site of course)

cd /usr/local/etc/letsencrypt/live/$SITE
cat fullchain.pem privkey.pem > /usr/local/etc/ssl/haproxy.pem
macon% cat /usr/local/etc/letsencrypt/renewal-hooks/post/002-haproxy.sh
#!/bin/sh
service haproxy restart

	  

When certbot renew is run, certbot checks the renewal-hooks/post directory and runs any executable things in it after it's renewed the certificate(s). As a side note, make sure you hit those scripts with chmod +x or they probably won't run.

Now all that's left is dropping a job into cron or periodic to run certbot renew at least once or twice within the renewal period.


macon% doas crontab -l|grep certbot
# certbot renewal
@monthly certbot renew

	  

You can always test that your scripts are working with certbot renew --dry-run just to be safe.


macon% doas certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /usr/local/etc/letsencrypt/renewal/53hor.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Simulating renewal of an existing certificate for 53hor.net and 7 more domains
Performing the following challenges:
http-01 challenge for 53hor.net
http-01 challenge for carpentertutoring.com
http-01 challenge for git.53hor.net
http-01 challenge for nextcloud.53hor.net
http-01 challenge for pkg.53hor.net
http-01 challenge for plex.53hor.net
http-01 challenge for theglassyladies.com
http-01 challenge for www.53hor.net
Waiting for verification...
Cleaning up challenges

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/usr/local/etc/letsencrypt/live/53hor.net/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all simulated renewals succeeded:
  /usr/local/etc/letsencrypt/live/53hor.net/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Running post-hook command: /usr/local/etc/letsencrypt/renewal-hooks/post/001-catcerts.sh
Running post-hook command: /usr/local/etc/letsencrypt/renewal-hooks/post/002-haproxy.sh
Output from post-hook command 002-haproxy.sh:
Waiting for PIDS: 15191.
Starting haproxy.


		

And there it is. Automated Let's Encrypt certificate renewal on FreeBSD with HAProxy.