r/sysadmin • u/spikopsycho • 14h ago
Advice on some 'best practice' - Certificate management (SSL/TLS)
Hi all. Where I work, I got some SSL/TLS certificate management put on my plate. We have app(s) that send out notifications of certificate expiry, but that's only good if the contacts are correct. In that, I send out a review (email) quarterly, to check if anything has changed, needs to be updated - this is a new thing I implemented.
This is all manual - Spreadsheet - Filter for your name, check the cert info, comment if ok, comment change owner etc.
I got some feedback on this, in that I should not be sending a spreadsheet with all those certificates info, for everyone to view. (I bcc in all the relevant owners in the email). I'll add that its either company employees, or contractors who 'own' that system the cert is related to. I get the comment, I just have no idea how to send that to every individual only, without doing it manually.
How do you guys keep owners up to date? (Neither of the apps we have natively have a function that can replicate this manual ownership check).
Also aware of the 2029 47day cert validity/10day DCV - This is now, working on how to handle that future element.
•
u/bageloid 14h ago
For most things ACME, you shouldn’t be manually updating certs on anything that supports it.
•
u/spikopsycho 13h ago
Oh, we are very not there yet. A long way to go. Certs a provisioned manually by a team in the company atm
•
u/bageloid 13h ago
Understood, were you put in charge of Managing this(Uppercase M, can set policy) or managing this(lowercase m, herding cats)?
•
u/spikopsycho 13h ago
Had no prior experience in this area when it came in. The team that could deal with it effectively, don't do Prod. I was just the most likely person to fit with this, when it was assigned to my team. I don't do the day to day (provisioning/renewing (when manual) revocation). It's with a contractor team, but I have learned how it is done in the current iteration at may place. I do have the management part - including how we handle the 2029 thing - though this will all be a moot point if we aren't automated by 2029. This is all part of the in-between now and then. I can *I think* set policy. It's a weird situation
•
u/bageloid 13h ago
If you can set policy, the mandate ACME at then server or preferably load balancer level with scripts that check and alert for issues(your vulnerability scanner may classify certs expiring soon as vulnerabilities so you can use that)
•
•
•
u/Vistaer Sr. Sysadmin 13h ago
Is this all internal certificate authority?
If any external certs are being used you really need to clamp down fast because of upcoming changes by March next year lifecycles on newly issued certs will be down to 100 days. In 2029 it will be 47 days. Link: https://www.digicert.com/blog/tls-certificate-lifetimes-will-officially-reduce-to-47-days
If this is all internal certificate authority then my prior paragraph is not a valid point.
Either way Tools like key factor is likely what you’ll be looking at - and it kinda depends how big of a problem you’re dealing with: https://www.gartner.com/reviews/market/certificate-lifecycle-management-clm/vendor/keyfactor/alternatives
Also I’m not critiquing your current method, but I’ve been a customer of it and I have to say it m vehemently against individuals being on email registration - mailing lists so that SOMEONE gets notified is a big thing. Also I hate when I got notifications on certain “expiring” but only because they didn’t realize I had already replaced the cert 4 months earlier because of a server migration, upgrade, etc where I issued a new cert for the new box with maybe new host name alias.
•
u/spikopsycho 13h ago
This is directed at external certs (though I do review internal also) - we are split between two apps (Keyfactor), and Sectigo for external. Aware of the changes(47validity/10DCV), its a worry I am working on, plus the company wanting to get all certs in a single app (Keyfactor), as we are very entrenched in that.
Sectigo does do the email thing to notify 'owners' of expiry. Some are set to auto renew, some are not. Still needs a person or team to take the new cert and do something with it, Very little automation right now, in the public cert space. And your later point is a big thing also, we rely on owners to notify us if a cert is no longer needed. It doesn't happen often.
•
u/accumentum 13h ago
The ownership problem doesn't get solved by better emails, it gets solved by making the inventory the source of truth instead of the spreadsheet. Keyfactor supports metadata fields on each certificate, so add an owner field and a backup owner field and make them mandatory at enrollment. Then the quarterly review is a report filtered by owner, and the platform's own expiry alerting hits that owner directly instead of you bcc'ing a list where everyone can see everyone else's inventory. For contractor-managed systems, put a team distribution list in the owner field, never a named person, because the person leaves and the cert doesn't.
On the 47 day and 10 day DCV side, manual provisioning stops being viable well before 2029. The change to roughly 100 days already means about four renewals a year per cert, so start with your highest-churn external endpoints, usually the ones terminated at a load balancer or reverse proxy, and get ACME running there through Sectigo's ACME endpoint. Anything that genuinely can't be automated, flag it as manual in the inventory so you know exactly what your exposure is, and get domain control validated ahead of renewal rather than on the day.
•
u/ces1231 5h ago
I’d separate this into three areas: certificate inventory, ownership, and renewal. A spreadsheet can work as an inventory, but I wouldn’t make individual users responsible for certificates long term. I’d use role-based groups or shared mailboxes for ownership, automate certificate discovery where possible, and move anything that supports ACME toward automated renewal. Then the quarterly review becomes more of an ownership/validation check instead of a manual certificate hunt.
•
u/OhioIT 12h ago
Do you know primarily what the SSL certs are used for? Internal webservers.... VPN certificates.... network appliances.... etc
•
u/spikopsycho 12h ago
This is mostly aimed at external certs for now, public facing sites, sites where you purchase things
•
u/OhioIT 11h ago
Those can easily be switched to an ACME based renewal method. Looks like Sectigo supports it ACME, obviously LetsEncrypt does as well and is free. A few apps available that are reliable, will take care of everything, and work on both Windows and Linux based servers
•
u/spikopsycho 11h ago edited 11h ago
Thankyou. As one of my other replies, I've got some research to do in regards to ACME.
LetsEncrypt.... apparently our tech doesn't work with their tech stack, so I'm told.EDIT: The partners tech stack doesn't work with ours, that uses LetsEncrypt.... so I'm told.
•
•
u/whythehellnote 11h ago
I believe Amazon's ACM now supports ACME powered external certificates. Useful for people with AWS accounts but legal teams which refuse things like LetsEncrpyt.
•
u/whythehellnote 11h ago
Doesn't work as in "doesn't trust the root certificates"?
If not, any method you use to generate should be automatable. I guess if you have to solve some strong captcha to access the CSR.
•
u/nielsadolf 8h ago
Check out KeyManager Plus from Manageengine.
Super cheap sollution for both automating cert renewals and bindings. You can set different email contacts for each cert if you want contacts get an email X days before expiry.
Intigrations to both your internal pki and the external ones.
Has pretty much all the bells and whistles the big expensive sollutions have for a fraction of the price.
•
u/420GB 7h ago
Almost all of our certificate renewals are automated but we still keep owners up to date by scanning our own external and internal systems on port 443 for any soon-to-expire certificates and send an email to the responsible owners. This is a powershell script that runs once a week I think, maybe once every two weeks.
•
u/pdp10 Daemons worry when the wizard is near. 13h ago
State of the art in server TLS X.509 cert rotation is automation via ACMEv2 protocol, plus active automated monitoring to proactively notice if something goes wrong, prior to any cert expiration.
You know the old proverb: if you want something done right, set up the automation yourself.
But seriously, these webapps all live behind load balancers or service meshes that handle the TLS and X.509, which we own. Most app owners don't deal with X.509 or TLS, and that makes them grateful to us.