talkgroup

Let's Encrypt warning, auto-renew script

Tags: #<Tag:0x00007fe166c945b0>

I set up a cron job to renew the cert every week. Today I got an email saying the cert would expire in 19 days. I thought that was strange, so I checked the logs:

/var/log/letsencrypt# tail letsencrypt.log
2019-02-17 07:23:00,419:INFO:certbot.renewal:Cert not yet due for renewal
2019-02-17 07:23:00,420:DEBUG:certbot.renewal:no renewal failures
2019-02-17 12:13:00,369:DEBUG:certbot.main:certbot version: 0.23.0
2019-02-17 12:13:00,370:DEBUG:certbot.main:Arguments: ['-q']
2019-02-17 12:13:00,370:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2019-02-17 12:13:00,380:DEBUG:certbot.log:Root logging level set at 30
2019-02-17 12:13:00,381:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2019-02-17 12:13:00,390:DEBUG:certbot.plugins.selection:Requested authenticator <certbot.cli._Default object at 0x7fbe41d77048> and installer <certbot.cli._Default object at 0x7fbe41d77048>
2019-02-17 12:13:00,399:INFO:certbot.renewal:Cert not yet due for renewal
2019-02-17 12:13:00,400:DEBUG:certbot.renewal:no renewal failures

I think everything is okay. I’m making a note here, in case in 20 days we can’t connect. :slight_smile:

I had an issue come up with some of my cert renewal scripts where using the “certonly” only command to request renewal, they would return “Cert not yet due for renewal” when they were in fact due. “Renew” was able to renew them however. Wasn’t sure of the cause, was just happy to have a solution at the time.

Just mentioning this if it’s something similar.

2 Likes

Funny, I mentioned it at How to handle SSL, Lets Encrypt when moving hosting, but I didn’t include the cron job here:

0 4 * * 0  /usr/bin/certbot renew --renew-hook "prosodyctl --root cert import /etc/letsencrypt/live" --quiet

@trashHeap is your’s similar?

Mine is a little different. I do a rolling check for all possible renewals once a week at 4am my timezone and any which need renewals get renewed via the webroot module.

I have seperate tasks to restart various services like prosody at five minute intervals thereafter. 4:05 4:10, 4:15 etc. etc. Prosody and lighttpd share SSL certs, so I do other things at those times like concatenate the fullchain and privkey files for lighttpd into a single file and so forth too. (Prosody can read both files seperately and I believe I have it set to now. Lighttpd might be able to too, but it expects a single file by default.)

Functionally though it’s still “certbot renew” tho. Im just restarting prosody five minutes later instead of calling prosodyctl right then in the renew-hook. I doubt renew-hook is causing the issue.

Definitely a different issue than what I was having.

1 Like

I just checked and my cert is running fine. I guess something isn’t triggering with Let’s Encrypt, so they are warning me the cert is about to expire, even though I check once a week and renew as needed. Hmmm.

You moved from one server to another right, and just had let’s encrypt reissue certs for the new server?

Is it possible their warning you about the old server’s certs?

Nope. New domain, new certs.

When I manually ran the commands it said I was good until May and didn’t need to renew yet. So, I’m gonna trust that until reality provides an alternative. :slight_smile:

1 Like

Hunh wierd.

Yeah you caused me to check out the mage.party SSL certs on the XMPP observatory and they look good till may.

I’ll copy the email next time. That might assist from that angle. :slight_smile:

Okay, the cron job runs fine. The certs are updated and imported. And yet, I still couldn’t connect to the server after the last one expired…

For whatever reason, I have to restart the Prosody service before it picks up the new certs. Though I just realized I didn’t just try reloading the config, maybe that clears a cache or something… I’ll wait until next cycle, and try reloading. If that works, I’ll just amend the cron job to do the same.

ah! thanks. i still can’t connect right now. next cron tonight?

I’ve not tried reloading the config but I always do a hard restart of prosody after updating the certs. I have cron restart prosody for me when it checks for cert renewal. I find that prosody seems to cache the old cert in memory.

1 Like

No, I restarted. It should work. Your clients may not reconnect well (common issue with clients).

1 Like

Had this issue again, but may have figured something out.

I ran that command, and among the output was:

The following certs are not due for renewal yet:
  /etc/letsencrypt/live/dump.mage.party/fullchain.pem expires on 2019-10-03 (skipped)
  /etc/letsencrypt/live/chat.mage.party/fullchain.pem expires on 2019-10-03 (skipped)
  /etc/letsencrypt/live/proxy.mage.party/fullchain.pem expires on 2019-10-03 (skipped)
  /etc/letsencrypt/live/mage.party/fullchain.pem expires on 2019-10-30 (skipped)
No renewals were attempted.
No hooks were run.

I suspect when it actually renews, it times out and never runs the hook, and never runs it under any other condition. I’ll need to figure out a better way, probably split it into two commands, if that is safe.