Add ACME DNS Authenticator?

[Bravo271]

Hi,

I've seen that the ACME DNS challenge is built into the FreeNAS GUI which is very nice.
However, currently there is only one provider available: "Route53"

I don't know which ACME client FreeNAS uses, but acme.sh can use APIs of many providers including INWX.

Is it possible to add another provider (e.g. INWX) manually to the GUI menu?

I've also found this article which I guess will work but having it directly in the GUI would be very comfortable. :)

 

[Basil Hendroff]

Join the queue. NAS-104912

 

[Bravo271]

Yeah but is it possible to add entries manually until this feature gets improved?

 

[Basil Hendroff]

Yeah but is it possible to add entries manually until this feature gets improved?

Unfortunately, no.

 

[proto]

looking at pip list soon or later...

certbot-dns-cloudflare 0.38.0
certbot-dns-digitalocean 0.38.0
certbot-dns-google 0.38.0
certbot-dns-ovh 0.38.0

 

[Bravo271]

Leaving the GUI aside, the certificates should be stored anywhere on the FreeNAS installation. (Where?)
Couldn't I just replace these with my newly generated ones?

Log in via SSH and use the acme.sh client from there?

I've did the same with my Proxmox installation and it worked fine there:

ACME with DNS challenge?

Hi, I want to generate Let's Encrypt SSL certificates for the Proxmox host and also for each service in my VMs. The ACME client is built into the Proxmox GUI but as far as I can see it can only use the http challenge to request certificates? And since my server is in a local network I don't...

forum.proxmox.com

 

[proto]

Couldn't I just replace these with my newly generated ones?

Certs go to freenas.db first...
So you have to use GUI or APIs.
You could use danb35's script that uses API v1.0 and it's working!

I use a "cert machine" to get certs from LE (digitalocean dns) and then deploy across my other servers including FreeNAS instances.

 

[Bravo271]

And how do you deploy? Via SSH & script?

Oh totally forgot about the script... yeah I'll try that. :D

 

[proto]

Basically it's working like this:
- get certs via certbot (I use wildcard certs)
- deploy via Ansible on basic servers (until my script is finished).
- use a custom python script to deploy on my NAS boxes (basically is the same script mentioned above)

Yes. On basic servers you need an SSH connection (paramiko or asyncssh if you use a python script) to deploy on servers and perform a "raw" copy of your live certificates, then restart the webserver.
On FreeNAS you need to use APIs because configuration is read from an sqlite DB. In case of a restart/upgrade you would loose the changes made via shell.

 

[danb35]

Couldn't I just replace these with my newly generated ones?

Of course you can:

Let's Encrypt with FreeNAS 11.1 and later

Integrating Let’s Encrypt TLS Certificates with FreeNAS FreeNAS has long had the ability to use HTTPS for the web GUI, but that has usually meant dealing with self-signed certificates and the associated headaches, or paying for a commercial...

www.ixsystems.com

 

[danb35]

So you have to use GUI or APIs.

Well, you don't have to. You can just copy the certs directly. It's an ugly hack, and completely unnecessary since the API supports the appropriate methods to do it properly, but it's still possible to do it without involving either the GUI or the API.

 

[Bravo271]

Yes I've already found your article and mentioned in the 1st post.
I'll give it a try later.

Thanks for that already. :)

 

[proto]

but it's still possible to do it without involving either the GUI or the API.

I've been too much excessive : - P

 

[Bravo271]

Should I install/run acme.sh in the FreeNAS OS or is it a good idea to create a separate jail for this?

I'm asking because I need 2FA to login to my DNS provider and that requires me to install oathtool.
And after some research how to install a package on FreeBSD or FreeNAS (Yes, I'm completely new to BSD... :D) I've found out that it isn't a good idea to install packages on the FreeNAS OS itself but I should use jails instead.

 

[danb35]

Should I install/run acme.sh in the FreeNAS OS or is it a good idea to create a separate jail for this?

It's a script, and there's no reason it can't be safely installed and run under the base OS. You may, however, want to make a point of installing it onto your pool rather than in /root--/root lives on the boot device, and if that device dies, so does your acme.sh installation. There's no reason you can't put it in a jail, but I don't see any reason to either, as long as you're using DNS validation.

I've found out that it isn't a good idea to install packages on the FreeNAS OS itself but I should use jails instead.

In terms of installing packages, that's good advice. Not all software needs to be installed from packages, though--some (like acme.sh) is just a script or collection of scripts, and can safely run just about anywhere. Other software, like rclone when downloaded from rclone.org, is a simple static binary that can also be put anywhere. In both of those cases, a jail is unnecessary, though you may still want to use one for other reasons.

 

[Bravo271]

Okay thank you. Installing it on the pool is a good idea.
How would I do that? Just cd to /mnt/pool/bla?

I guess then I better should get my permissions right.
Thats another thing I really need to read into... currently ZFS with the datasets and permissions is a little bit confusing to me.
(Also completely new to ZFS.. :D)

But for now I think its fine to put on /root/ since FreeNAS is a VM in Proxmox which is stored on a ZFS mirror of SSDs.
So it shouldn't get lost. I guess...

To use 2FA for the DNS challenge, the acme.sh script told me to install "oathtool".
That worked pretty fine on Debian, but as far as I see this package isn't available on FreeBSD.

Have you ever used 2FA here?

 

[danb35]

Just cd to /mnt/pool/bla?

acme.sh --install --home-dir /blah, IIRC. Check its documentation to make sure I have the correct option for the home directory.

FreeNAS is a VM in Proxmox which is stored on a ZFS mirror of SSDs.

To use 2FA for the DNS challenge

I'd missed that this was needed for your DNS host--in that case, it'll need to go in a jail along with acme.sh. The FreeBSD package appears to be oath-toolkit, which you'd install using pkg install oath-toolkit.

 

[Bravo271]

I know about the "danger" but I've passed though a PCIe HBA to the FreeNAS VM so it has direct access to the drives.
Should be fine. But we'll see. I've backups of everything and it's just a hobby project, no production environment. :)

So if I want to install something with pkg I need to put it in a jail?
That's the rule of thumb?

ahh oath-toolkit. Thanks. That is useful information.

 

[danb35]

So if I want to install something with pkg I need to put it in a jail?
That's the rule of thumb?

Yes. Other software might be installable outside of a jail (though if you aren't sure, inside a jail is always safe), but it isn't possible to install packages in the base OS without tinkering with things, which you shouldn't do.

 

[Bravo271]

Okay I just played around and found out that it isn't as easy as described here if I want to put everything in a jail.

To make everything work I had to install some packages first:

Code:

pkg install wget bash nano python3 py37-pip oath-toolkit ca_root_nss
python3 -m pip install requests

chsh -s /usr/local/bin/bash
echo 'cd ~' >> ~/.bashrc

wget to install acme.sh and your script. (I didn't want to install git for that since it is 300MB+)
bash just for convenience since I'm not familiar with csh
python3 and py37-pip because python and requests module is required by your script.
oath-toolkit to use 2FA with acme.sh
ca_root_nss to wget via https

I don't know why but acme.sh added source "/root/.acme.sh/acme.sh.env to .zshrc instead of .bashrc even if I set bash as default shell and re-login. So I did had to do that manually.

Then your script returned the following:

Code:

[Mon Apr 20 17:43:16 CEST 2020] Run reload cmd: /root/deploy-freenas-master/deploy_freenas.py
Traceback (most recent call last):
  File "/root/deploy-freenas-master/deploy_freenas.py", line 44, in <module>
    PASSWORD = deploy.get('password')
  File "/usr/local/lib/python3.7/configparser.py", line 1301, in get
    fallback=fallback, **kwargs)
  File "/usr/local/lib/python3.7/configparser.py", line 799, in get
    d)
  File "/usr/local/lib/python3.7/configparser.py", line 394, in before_get
    self._interpolate_some(parser, option, L, value, section, defaults, 1)
  File "/usr/local/lib/python3.7/configparser.py", line 444, in _interpolate_some
    "found: %r" % (rest,))
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%'
[Mon Apr 20 17:43:16 CEST 2020] Reload error for :

Probably because I had a lot of special characters in my FreeNAS password.
I changed it to only letters and numbers... but that's a bad solution tbh.

That wasn't the last problem. Then I received the following:

Code:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 370, in connect
    ssl_context=context,
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 377, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/ssl.py", line 423, in wrap_socket
    session=session
  File "/usr/local/lib/python3.7/ssl.py", line 870, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.7/ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1076)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='nas.local.domain.tld', port=443): Max retries exceeded with url: /api/v1.0/system/certificate/import/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1076)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/deploy-freenas-master/deploy_freenas.py", line 73, in <module>
    "cert_privatekey": priv_key,
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 514, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='nas.local.domain.tld', port=443): Max retries exceeded with url: /api/v1.0/system/certificate/import/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1076)')))
[Mon Apr 20 17:49:21 CEST 2020] Reload error for :

That was because I've set verify = true in deploy_config.
But I was still using the default FreeNAS self signed certificate.

I've set that to false and then finally everything worked.
I guess now I can set verify = true again for future renewals.
Shouldn't be a problem anymore.

But now I have one last question @danb35
You wrote that I should set a cronjob for automatic renew in the FreeNAS gui.
But why? acme.sh already installs the following cronjob by default:

Code:

[root@acme ~]# crontab -l
15 0 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" > /dev/null