TLS certificates with TrueCommand

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
As I've been able to use Let's Encrypt to "encrypt all the things!", attention naturally turns to TrueCommand. As it's in a Docker container with a persistent data directory, it would seem like a Let's Encrypt client could put the cert there for use, but the manual doesn't mention that as a possibility. In fact, it only mentions manually uploading the cert through the GUI. Is that really the only way to apply a cert to a TC installation? Edit: is it possibly as simple as dropping the new files into /data/truecommand/ ?

Yes, I know about a reverse proxy--I'm talking about the TC installation handling the TLS termination itself.
 
Last edited:

kenmoore

TrueCommand Project Lead
iXsystems
Joined
May 1, 2019
Messages
51
Yes, with a caveat:
You can manually drop your custom cert files into the /data directory for the TrueCommand container if you follow these steps:
1. Make sure you use the ".custom" suffixes on the files
* [datadir]/truecommand/server.key.custom
* [datadir]/truecommand/server.crt.custom
2. Restart the docker container
* This will detect those files and re-load them into the proper places within the container itself to make them "live".
* If you use the TrueCommand UI to import the files, then it dynamically triggers updates for everything inside the container, so no restart is needed.

Additional option: Use the TrueCommand API!
This is exactly what the TrueCommand UI does in the background, so any certs submitted this way will get loaded/used instantly:

API reference: https://api.ixsystems.com/truecommand/api/ssl/cert_import/
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
Additional option: Use the TrueCommand API!
That's what I'd been kind of hoping for, but hadn't yet seen the docs. So something like this should do the trick:
curl -l -g --data '{"args" : { "pem" : "$(cat fullchain.pem)", "key" : "$(cat privkey.pem)" } }' -u "username:password" -X GET http://[IP_ADDRESS]/api/ssl/cert_import

Edit: well, it returns this:
Code:
{
    "result": "success"
}

so I guess it does work. Now to get acme.sh to use that whole command as a renew hook.

Edit 2: it's kind of interesting that authentication is with username/password rather than API tokens. Is there any thought of changing to these? Seems they would have the potential of being more secure.
 
Last edited:

ornias

Wizard
Joined
Mar 6, 2020
Messages
1,458
I personally think this is rather complex way if doing letsencrypt with docker.
For example: I run a (single node) docker swarm, everything sits behind a traefik reverse proxy.
Once setup I can spin up every gosh damn container I want and everything gets A+ (SSL labs) SSL certs using letsencrypt like magic.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
I personally think this is rather complex way if doing letsencrypt with docker.
Different strokes, different folks. To me, your method sounds much more complex. "Get cert for service, install cert on service" seems pretty straightforward, particularly if "install cert on service" can be automated with a single command (as it can be). No need for a reverse proxy, and I'll admit I don't know enough about Docker to know what a "docker swarm" is, much less why I might want one.
 

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
@danb35 -- Did you ever figure this out with a acme renewal hook? I looked as the restful API reference -- and frankly I didn't see anything like what you listed in your command for cert_import. I'd really like to avoid putting a reverse proxy in front of installation. I'm running True Command docker image. I'd like to renew the SSL certs on the host system using acme.sh (linux host), and then post renewal auto import the certs to true command.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
looked as the restful API reference -- and frankly I didn't see anything like what you listed in your command for cert_import.
Well, there are two problems: (1) the link up-thread is no good any more, and (2) the new documentation site seems to be severely broken to the point of being near-unusable. The new link is https://www.truenas.com/docs/hub/truecommand/tc_api/; good luck finding anything there though. I think your problem is (as mine was for a few minutes just now) that you're looking at the docs for the FreeNAS/TrueNAS API, not the TrueCommand API.

But yes, I have it working just fine; my script is really just a refinement of what I posted above. You'd obviously replace fullchain.pem and privkey.pem with full paths to those files, or with variables that contained those paths; and username:password with the actual username and password.
 

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
@danb35
It would honestly be really great to pass the fullchain.pem and privkey.pem files as bind mounts to the container. In addition it would preferable to pass the username and password as docker secrets. I'm sure you were hinting at these recommendations in your first post however. Honestly I don't really need the GUI to reflect the loaded TLS credentials, its kind of useless.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
It would honestly be really great to pass the fullchain.pem and privkey.pem files as bind mounts to the container.
In that case, you have instructions up-thread too: put them into data/truecommand/server.key.custom and data/truecommand/server.crt.custom and restart the Docker image. And in that case, since you aren't using the API, you don't need to authenticate to it. The file copy and image restart could both be handled by acme.sh.
I'm sure you were hinting at these recommendations in your first post however
Not really; I don't know enough about Docker to think to suggest that. But there's lots of ways that could be used to make the API call which wouldn't involve specifying the credentials on a command line.
 

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
@danb35 -- Do you know much about docker? Should certificates be passed as bind mounts or docker secrets? Once certificates are renewed does container need a restart to detect file changes? Usually with like nginx containers you could potentially shell into container and have nginx re-read config -- is that possible in this case or is it advisable to just restart container?
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
Do you know much about docker?
No, very little, in fact.
Should certificates be passed as bind mounts or docker secrets?
Based on responses up-thread, it seems they would need to appear at certain filesystem paths within the docker container. Beyond that, I couldn't say.
Once certificates are renewed does container need a restart to detect file changes?
Once again, based on responses up-thread, I'd say yes. Or just use the API, in which case the reload is handled automatically.
 

KevDog

Patron
Joined
Nov 26, 2016
Messages
462
@danb35

Ok thanks for information. I think we could agree passing credentials such as an SSL private key and password in plaintext through curl probably not the most "secure" way to do things. Particularly when docker has implemented workarounds (such as secrets) to address this specific issue since it comes up quite commonly when working with docker projects. I remember I contacted the maintainer of the docker watchtower module and I think he had a beta utilizing secrets within a few days and I know it's made it into the main source code by now. Other docker projects and conversation with other docker developers kind of keyed me into these methods.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
passing credentials such as an SSL private key and password in plaintext through curl
There's a simple way to address that: make the API connection via HTTPS. But really, where's the risk in a connection to http://localhost? I don't like the idea of storing the password in plaintext (even if obfuscated plaintext) in the renew-hook command or however it would be scripted, and I think I'd rather see an API key used, but the private key is going to be stored that way anyway. Filesystem permissions are deemed adequate to protect it on 99.99% of systems.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
So something like this should do the trick:
curl -l -g --data '{"args" : { "pem" : "$(cat fullchain.pem)", "key" : "$(cat privkey.pem)" } }' -u "username:password" -X GET http://[IP_ADDRESS]/api/ssl/cert_import
Well, it did--until last November or so, and because I don't use TrueCommand very much, I didn't notice it very much. But a problem it remains--now, running the same command (and, naturally, with full paths for the cert/key files, and actual login credentials), TrueCommand returns this error:
Code:
{"error":"invalid character '\\n' in string literal"}#

This is now running TrueCommand 2.1.1--looks like there's been a change in the relevant code from 16 Sep 21 (the time the last installed cert was issued) and 16 Nov 21 (when the cert was renewed, but installing the renewed cert failed).

Interestingly, it appears uploading a cert through the GUI also fails, and silently. The upload works without error, and the new cert is saved to disk--but it's still serving the old (and in this case, long-expired) cert.
 
Last edited:

zwf

Dabbler
Joined
Aug 6, 2017
Messages
35
Interesting....
i am using your deploy freenas to update my cert to my truenas and works well.
wonder if this issue you describe has been fixed in 2.2 ?

there is a few bug in 2.1.1 who suppose to have been fixed on 2.2 but waiting for release or ,is there a way to test on a prerelease version on TC 2.2 ?
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
there is a few bug in 2.1.1 who suppose to have been fixed on 2.2 but waiting for release
I'm pretty sure those are my bugs. Yes, supposedly this is (will be) fixed in 2.2. I guess we'll see, since the comments on the bug indicated they couldn't duplicate the problem in the first place.
 
Top