Resource icon

TLS certificates from a local certificate authority

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,464
danb35 submitted a new resource:

TLS certificates from a local certificate authority - TLS certificates from a local certificate authority

A few years ago, I wrote a resource with an accompanying script for obtaining a certificate for your FreeNAS server from Let's Encrypt. The script has worked well for me, with some updates, since then.

More recently, I've come to think that, for LAN resources that will only be used by local clients, a local certificate authority would make more sense. I ran across this site:
[URL...

Read more about this resource...
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
It was with some trepidation that @danb35 accepted the challenge of having a beta tester like me. As the clown troupe pulled into the parking lot and emerged from the impossibly small car, I took it on myself to make lots of mistakes that will hopefully make the script and the process that much more resilient and more foolproof for the rest of you.

Unlike the traditional let’s encrypt process (see @danb35's original entry and the jail building script by @Basil Hendroff), this approach does not require a jail. In fact, you don’t want a jail or you won’t be able to replace the proper nginx file. I installed the two scripts in my /root directory using the very easy-to-use git commands documented on @danby35 two github pages to make the certificate and then to deploy it.

Hardware Prerequisites:
For one, the internal CA. There are many options but I chose the Raspberry-Pi based solution described at smallstep because it allowed me to reuse an older Pi 3 model (no speed records being broken here). In terms of hardware, they recommend a Raspberry Pi with 2GB of RAM, a Yubikey 5 NFC to securely store the certificates, a infinite noise TRNG, and a small backup flash drive. I also recommend using a so-called "high endurance" micro SD drive usually meant for cameras.

Network Setup:
Another good step is assigning your network devices names as well as either static IPs or quasi-static IPs (by having your DHCP server reserve only one IP address to them). I use a mirror pair of raspberry Pis with Pi-Hole to serve as secure DNS servers and a Edgerouter to assign quasi-static IP addresses. Between hosts files and the "local DNS" entries on the Pi-holes, every internal address resolves nicely when I run a nslookup.

Internal CA setup:
Unless you are familiar with all the nomenclature, the setup process, etc. then expect this process to take several hours. You are installing a lot of stuff on a little Pi (which takes time) and you should be very clear about how your internal network is set up. For example, my initial device names used ".home" as a domain, which I later changed to ".internal" since ".internal" is less likely to be sold off and commercialized.

Some minor suggestions/thoughts re: the build process:
  • The setup guide at smallstep is amazingly complete but I suggest setting up the PKI and PUK on the Yubikey in advance. To me, the Yubikey application is a lot easier to use than the RPi CLI.
  • A password manager is a must. A lot of passwords / hashes / etc. will be thrown at you.
  • Once your Internal CA is running a ACME server successfully, I suggest you shut it down, then back up the entire microSD card.
  • Consider enabling auto-update (see here)
  • Rather than stop all SSH access to the internal CA, consider using passwordless SSH to make said access quite secure yet still easy-to-use.
Using the scripts:
As for @danb35's script, I really did my best to be a better idiot. I broke it in many creative ways... and then @danb35 would modify his script accordingly. The recovery readme's in the Github repo are also a testament to the technical support he had to give me to revive my "403- forbidden" locked up TrueNAS UX when my interruptions caused the original nginx backup file to get nuked. So, I didn't manage to brick my NAS, just the UX, which is what the console option 11 is for. That said, do not interrupt the scripts if stuff isn't working, wait for the script to naturally terminate. Use the -d argument to enable debugging / verbose mode to make trouble-shooting easier.

Of the two scripts, the first one (freenas-nginx-swap) is a bit trickier to set up. If stuff breaks, chances are you may have a misspelling or a bad path. Also, you need to SCP or otherwise transfer the root certificate from your internal CA to your freeNAS. I deposited mine in the freenas-nginx-swap folder.

If your are running TrueNAS, use the API Key tool (see gearset in the top right corner of the GUI) to create a API key to use as a password for step 2, actually deploying the certificate. This is actually a really good reason to upgrade to TrueNAS if you want to use an internal CA, since the FreeNAS version of the script has to store your root password in plaintext. (!!!)

If both scripts work as intended (and they will only work properly if you generate a new certificate followed by a deployment) then set up a cron job to execute them automatically as needed. @danb35 suggests running them one after the other by scheduling only one cron job (See Tasks -> Cron) and then using && to run both scripts with one cron job.

If anyone needs help with getting the internal CA to work with Win-ACME, I can now likely give some pointers. I heartily recommend Win-ACME to anyone who needs a Windows-based ACME client.

Lastly:
A huge thank-you to the folk at Smallstep to document the internal CA build process as much as they did. Similarly, a huge thank you to @danb35 for putting in all the time and expertise to create these scripts, document their use, and the later modifications to make them more resilient. Additionally, I could not have done it without his kind and timely tech support when I managed to mess something up.
 
Last edited:
Joined
Jan 4, 2014
Messages
1,644
Great effort getting this working @Constantin @danb35 and making it available to the TrueNAS community! It solves a myriad of problems and brings us one step closer to the end of self-signed certs.

I've also been trying to get a local CA working, but going the software only route. The smallstep CA has been ported to FreeBSD (pkg install step-certificates) and Caddy also uses the smallstep libraries so setting up an internal CA should be possible using either. However, there isn't any code at present to install the smallstep root certificate under FreeBSD.

You can feel the frustration, of installing additional certificates on FreeBSD, from the FreeBSD community in this FreeBSD bug report. This hasn't made it easy for smallstep developers, unfamiliar with FreeBSD, to solve the problem for the FreeBSD platform (and its derivatives like TrueNAS). It needs one final push to make this work, but it requires clear direction from individuals who understand the ins and outs of adding additional certificates to FreeBSD and making sure those certificates persist through upgrades. I haven't been able to advise with a high degree of confidence. If you are able to help move this along, please consider contributing to this smallstep trust store issue.
 
Last edited:

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
You raise a good point. Running a CA from within a TrueNAS could allow yet another network device / service to get consolidated into the NAS, with some benefits.

at the same time, between the Yubikey and the noise TRNG, the small step solution brings a level of security to the CA that is likely very difficult to replicate in FreeBSD without some serious software skills. I’m also a fan of not consolidating everything in one device to limit the impacts of individual device failure. Queen bee would not be pleased.

so that’s why I have a stack of RPis here, individually running services that likely could be consolidated into the NAS or fewer pi’s. After all, they are not stressed under current use. That said, the NAS might make for a good backup / mirror to add to redundancy for critical services, ie pi-hole #3 running in a jail.

Unfortunately, the only way I can help the folk writing this sort of software is via $$$ donations. My coding skills are very limited.
 
Last edited:

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,464
I've moved a lot of the stuff that I used to run on Pis to more permanent infrastructure, though mostly on a Proxmox PVE HA cluster rather than my TrueNAS box. But for a CA, I like the idea of it being a dedicated device, I like the idea of a hardware RNG, and I like the idea of the signing keys being stored in what's effectively a HSM (with an offline backup, but the keys never hit the system's root storage device). Do I "need" any of this? Not really--it's for my home LAN. But it does seem like the "right" way to do it, and it's still pretty inexpensive.

This is actually a really good reason to upgrade to TrueNAS if you want to use an internal CA
This is true even if you're getting Let's Encrypt certs for your server, as I've been doing for a while. In order to deploy the cert, it needs to authenticate to the API, and prior to version 12, you could only do that with the root password; starting with TN12, you can use an API key instead
@danb35 suggests running them one after the other by scheduling only one cron job (See Tasks -> Cron) and then using && to run both scripts with one cron job.
You can string commands together with && and with ;--but && will only run the next command if the first completes successfully (that is, with an exit status of 0).
For example, my initial device names used ".home" as a domain, which I later changed to ".internal"
I have a public domain that I've used for a number of years, so I'm using that for my local hosts. pfSense provides local DNS.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,464
Oh, one other thing I did when setting up the CA--realizing (from long experience on this forum) how crappy USB sticks are, I also used qrencode to generate a QR code of the keys--still on the USB stick. I then printed that, and put that printout in the safe with the USB stick. And since the safe is one of these:

...I think it'll be secure enough. And this way, even if the USB stick dies, I have a machine-readable backup of the signing keys--indeed, I should have that backup even if the house burns down, though I'm not sure I'll need it then.
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
setting up the Pi’s took hours. So I will be backing up those sweet little microSDs to ensure I don’t have to redo all of that again.

ditto the edgerouter config files, the NAS config, and so on. The backup drive for the internal CA is super easy to store , and I really like the idea of putting in a QR code also. put it all in a zip lock and it’s sealed and protected.
 
Joined
Apr 27, 2023
Messages
1
I know this is reviving an older post but firstly I wanted to say thank you for the scripts as I am in the same boat, self-hosted Smallstep CA. It has helped immensely and I didn't have much trouble using them, except as Constantin had mentioned... typos.

Second of all are a couple questions in regards to renewing.
It appears that the way you are implementing is to recreate a new certificate from scratch each time instead of using the acme.sh renew command, correct?
If so, thoughts on using acme.sh --renew? (I understand script changes would need to be made)
How long are you setting the certificates?
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,464
It appears that the way you are implementing is to recreate a new certificate from scratch each time instead of using the acme.sh renew command, correct?
That's correct. The reason is that acme.sh doesn't have very good granularity for when to renew a cert; you can only tell it to renew a cert n days before it expires. If the cert's only valid for 24 hours to begin with, which is the default with Smallstep's ACME authenticator, that just won't work. The way Smallstep recommend to deal with that is what I've done--just force-issue the cert at shorter intervals (they recommend every 8 hours, IIRC). The other option would be to configure the CA to issue longer-lived certs--maybe with a duration of four days--and tell acme.sh to renew at, say, two days.
 
Top