How to Automate Certbot Renewal with HAProxy
So this is specifically for HAProxy on FreeBSD, but it should apply to
other *nix systems as well. Basically, I use HAProxy as a reverse proxy
to a bunch of servers I administer. I use Let's Encrypt for a
certificate and I used certbot to generate that
certificate. Generating the certificate for the first time is easy and
has lots of documentation, but it wasn't initially clear on how I could
easily set up auto-renewal. Here's how I did it.
If you've already set up TLS termination with HAProxy and
certbot, you know you need to combine your Let's Encrypt
fullchain and private key to get a combined certificate that HAProxy can
use. You can do this by cat-ing the chain and key together
like so:
cat /usr/local/etc/letsencrypt/live/$SITE/fullchain.pem /usr/local/etc/letsencrypt/live/$SITE/privkey.pem > /usr/local/etc/ssl/haproxy.pem
In this example, $SITE is your domain name that you fed
HAProxy when you created the certificate and haproxy.pem is
wherever you're storing HAProxy's combined certificate. Your HAProxy
config then points to that certificate like this:
macon% grep crt /usr/local/etc/haproxy.conf
bind *:443 ssl crt /usr/local/etc/ssl/haproxy.pem
And that was the end of the first-time setup. Then a few months later
you probably had to do it again because Let's Encrypt certs are only
good for 90 days in between renewals. To renew the certificate, you
usually run certbot renew, it detects which certificates
are present, and uses either the webroot or standlone server renewal
process. Then you have to cat the fullchain and privkey
together and restart HAProxy so it starts using the new certificate.
To automate those steps, newer versions of
certbot will run any post renewal hooks (read: scripts)
that you want. You can also configure HAProxy and
certbot to perform the ACME challenge dance for renewal so
that you don't have to use it interactively.
First, if you haven't already done it, change your HAProxy config so
there's a frontend+backend for responding to ACME challenges. In a
frontend listening for requests, create an access control list for any
request looking for /.well-known/acme-challenge/. Send
those requests to a backend server with an unused local port.
frontend http-in
acl letsencrypt-acl path_beg /.well-known/acme-challenge/
use_backend letsencrypt-backend if letsencrypt-acl
...
backend letsencrypt-backend
server letsencrypt 127.0.0.1:54321
What this will do is allow certbot and Let's Encrypt to
renew your server in standalone mode via your reverse proxy. As an added
bonus it prevents you from having to open up an additional port on your
firewall.
Now you've gotta configure certbot to do just that. A
config file was created in certbot's
renew directory for your site. All you need to do in that
file is add a line to the [renewalparams] section
specifying the port you're using in your HAProxy config.
macon% echo "http01_port = 54321" >> /usr/local/etc/letsencrypt/renewal/$SITE.conf
Now you need the post-renewal hooks. I dropped two separate scripts into
the renewal-hooks directory: one does the job of combining
the certificate chain and private key and the other just restarts
HAProxy.
macon% cat /usr/local/etc/letsencrypt/renewal-hooks/post/001-catcerts.sh
#!/bin/sh
SITE=(your site of course)
cd /usr/local/etc/letsencrypt/live/$SITE
cat fullchain.pem privkey.pem > /usr/local/etc/ssl/haproxy.pem
macon% cat /usr/local/etc/letsencrypt/renewal-hooks/post/002-haproxy.sh
#!/bin/sh
service haproxy restart
When certbot renew is run, certbot checks the
renewal-hooks/post directory and runs any executable things
in it after it's renewed the certificate(s). As a side note,
make sure you hit those scripts with chmod +x or
they probably won't run.
Now all that's left is dropping a job into cron or
periodic to run certbot renew at least once or
twice within the renewal period.
macon% doas crontab -l|grep certbot
# certbot renewal
@monthly certbot renew
You can always test that your scripts are working with
certbot renew --dry-run just to be safe.
macon% doas certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /usr/local/etc/letsencrypt/renewal/53hor.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
Plugins selected: Authenticator standalone, Installer None
Simulating renewal of an existing certificate for 53hor.net and 7 more domains
Performing the following challenges:
http-01 challenge for 53hor.net
http-01 challenge for carpentertutoring.com
http-01 challenge for git.53hor.net
http-01 challenge for nextcloud.53hor.net
http-01 challenge for pkg.53hor.net
http-01 challenge for plex.53hor.net
http-01 challenge for theglassyladies.com
http-01 challenge for www.53hor.net
Waiting for verification...
Cleaning up challenges
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/usr/local/etc/letsencrypt/live/53hor.net/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all simulated renewals succeeded:
/usr/local/etc/letsencrypt/live/53hor.net/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Running post-hook command: /usr/local/etc/letsencrypt/renewal-hooks/post/001-catcerts.sh
Running post-hook command: /usr/local/etc/letsencrypt/renewal-hooks/post/002-haproxy.sh
Output from post-hook command 002-haproxy.sh:
Waiting for PIDS: 15191.
Starting haproxy.
And there it is. Automated Let's Encrypt certificate renewal on FreeBSD with HAProxy.