Let's Encrypt with FreeNAS 11.1 and later 0.3

[danb35]

danb35 submitted a new resource:

Let's Encrypt with FreeNAS 11.1 and later - Automatically obtain and update TLS Certificates from Let's Encrypt

Integrating Let’s Encrypt TLS Certificates with FreeNAS

FreeNAS has long had the ability to use HTTPS for the web GUI, but that has usually meant dealing with self-signed certificates and the associated headaches, or paying for a commercial certificate. With the launch of Let’s Encrypt in December 2015, trusted TLS certificates became available at no cost. However, their short lifetime, as well as the requirement to use other software tools to issue them, has caused some challenges in...

Read more about this resource...

 

[bobpaul]

It looks like this requires socat which isn't in FreeNAS. I ended up putting this in a jail where I could install socat with pkg.

 

[danb35]

That's what I get for writing without testing... The DNS method doesn't require socat, but it looks like standalone does.

 

[ovizii]

sounds great and I am using acme.sh for different other non-freenas related projects but isn't this going to break when the next freenas update comes around? I always thought we shouldn't be installing anything into freenas itself for this very reason?

 

[danb35]

isn't this going to break when the next freenas update comes around?

It shouldn't. acme.sh is a script that will live in the root user's home directory, which isn't affected by any upgrades. The cert/key/chain files are saved there as well. The script to deploy the cert uses the API, though I do need to adjust one API endpoint.

It should only break if there are major, incompatible changes to the API.

 

[ovizii]

Thanks for clarifying. I "assumed" one shouldn't touch freenas at all but I guess its better to ask ;-)

 

[danb35]

danb35 updated Let's Encrypt with FreeNAS 11.1 and later with a new update entry:

Use new API method to restart web server

It turns out that the API method that was supposed to restart the web GUI in FreeNAS 11.1 didn't work. The devs have added a new method with 11.1-U2, which this update uses. I've also moved the code to GitHub; the button will take you to its page, but the easiest way to grab the script itself is to do:
fetch https://raw.githubusercontent.com/danb35/deploy-freenas/master/deploy_freenas.py

Read the rest of this update entry...

 

[Stefan1970]

I've got an error with deploy_freenas. This is with no certs installed and using http protocol. Issuing a certificate works flawless, it always fails when calling the hook to deploy_freenas:

Code:

root@storage:~ # ./deploy_freenas.py  
Certificate import successful
Certificate list successful
Setting active certificate successful
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
	chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
	six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
	httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
	response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
	version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 266, in _read_status
	raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
	timeout=timeout
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
	_stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 357, in increment
	raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
	raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
	chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
	six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 383, in _make_request
	httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.6/http/client.py", line 1331, in getresponse
	response.begin()
  File "/usr/local/lib/python3.6/http/client.py", line 297, in begin
	version, status, reason = self._read_status()
  File "/usr/local/lib/python3.6/http/client.py", line 266, in _read_status
	raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./deploy_freenas.py", line 99, in <module>
	auth=(USER, PASSWORD),
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 112, in post
	return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request
	return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 502, in request
	resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 612, in send
	r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 490, in send
	raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

When I run it again:

Code:

root@storage:~ # ./deploy_freenas.py
Error importing certificate!
<Response [400]>

I have to change system\general\protocol to http+https and in deploy_freenas I also change the protocol to https. When I run deploy_freenas again it fails with:

Code:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 441, in wrap_socket
	cnx.do_handshake()
  File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1426, in do_handshake
	self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1174, in _raise_ssl_error
	_raise_current_error()
  File "/usr/local/lib/python3.6/site-packages/OpenSSL/_util.py", line 48, in exception_from_error_queue
	raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
	chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
	self._validate_conn(conn)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
	conn.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 326, in connect
	ssl_context=context)
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
	return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 448, in wrap_socket
	raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
	timeout=timeout
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
	_stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
	raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='storage.hijnn.net', port=443): Max retries exceeded with url: /api/v1.0/system/certificate/import/ (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./deploy_freenas.py", line 45, in <module>
	"cert_privatekey": priv_key,
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 112, in post
	return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request
	return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 502, in request
	resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 612, in send
	r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 504, in send
	raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='storage.hijnn.net', port=443): Max retries exceeded with url: /api/v1.0/system/certificate/import/ (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))

What am I doing wrong?

 

[danb35]

In the first example, is the cert installed? If so, though the output is ugly, it may be working as expected. The new API call appears to hard-restart the web server, which results in the connection error you're seeing.

The error the second time is also to be expected--you're trying to import the same cert (with the same name) a second time, which the system won't allow.

Can you browse to the web GUI using HTTPS?

 

[Stefan1970]

Yes the certificate is installed and I can browse the GUI using HTTPS.

I'll leave it for now. Just curious to see what happens when the cron job runs and my cert is 60+ days old.

 

[Benc]

Is it possibile I've made a mess quiting process in the middle (before certificate part) and just deleting files in /root? I am not sure, because it could also be because of latest update to U3, but I can't login to Windows shares any more. Also, if I want to download acme.sh (curl https://get.acme.sh | sh) i get error 'could not resolve host: get.acme.sh'
Is this somehow connected or should I look elsewhere?

 

[danb35]

I think there are two separate issues here. The Windows shares problem is definitely a -U3 issue. The "could not resolve host" might be different, but still shouldn't result from running this script. From the shell, what's the output of ping google.com?

 

[Benc]

I think there are two separate issues here. The Windows shares problem is definitely a -U3 issue. The "could not resolve host" might be different, but still shouldn't result from running this script. From the shell, what's the output of ping google.com?

I fixed samba problem and I can login now, and 'resolving' problem is also gone. So now I have acme.sh script installed again, is there a proper way to remove it?

 

[danb35]

So now I have acme.sh script installed again, is there a proper way to remove it?

• Delete /root/.acme.sh/
• Delete deploy-freenas.py
• Remove the cron job calling acme.sh

 

[Benc]

• Delete /root/.acme.sh/
• Delete deploy-freenas.py
• Remove the cron job calling acme.sh

I deleted acme.sh while deploy-freenas.py was not downloaded this time. But there is no cron job (neither was the first time). I checked Cron Jobs in GUI and /etc/cron.d folder is empty. Is that ok?

 

[danb35]

But there is no cron job

Setting up the cron job is the last step, but if it isn't there, then there's nothing to remove.

 

[Hazza]

I have a similar issue to Stefan1970.
I already have a valid Let's Encrypt certificate configured in the web UI - manually imported.
I am running FreeNAS-11.1-U4.
The FreeNAS nginx config is set to only allow connections using TLS 1.0 and above.

Code:

		ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

When the web UI is set to HTTPS, or HTTP+HTTPS, and the deploy_freenas.py script is set to use HTTPS, I get the following output:

Code:

root@freenas:~ # ./deploy_freenas.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 441, in wrap_socket
	cnx.do_handshake()
  File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1426, in do_handshake
	self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1174, in _raise_ssl_error
	_raise_current_error()
  File "/usr/local/lib/python3.6/site-packages/OpenSSL/_util.py", line 48, in exception_from_error_queue
	raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
	chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
	self._validate_conn(conn)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
	conn.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 326, in connect
	ssl_context=context)
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
	return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 448, in wrap_socket
	raise ssl.SSLError('bad handshake: %r' % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
	timeout=timeout
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
	_stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
	raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='freenas.mydomain.com', port=443): Max retries exceeded with url: /api/v1.0/system/certificate/import/ (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./deploy_freenas.py", line 45, in <module>
	"cert_privatekey": priv_key,
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 112, in post
	return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request
	return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 502, in request
	resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 612, in send
	r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 504, in send
	raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='freenas.mydomain.com', port=443): Max retries exceeded with url: /api/v1.0/system/certificate/import/ (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))

The use of the term 'ssl3_get_server_certificate' seems to suggest that the code is trying to connect using SSLv3, which I have denied.
Is this the case? Is it possible to configure the script to use TLS 1.0 or above instead?
If I configure the script to use HTTP instead of HTTPS, the certificate is deployed OK, albeit with the expected 'connection closed' error as the web UI is restarted.
I would, of course, prefer to use the script in HTTPS mode, even if it is just localhost communication, so that I can disable HTTP access to the web UI.

 

[Stefan1970]

Yes the certificate is installed and I can browse the GUI using HTTPS.

I'll leave it for now. Just curious to see what happens when the cron job runs and my cert is 60+ days old.

The time is here to update my cert.

The first time the cron failed with:

Code:

[Wed May 16 11:26:02 CEST 2018] Please add the TXT records to the domains, and retry again.
[Wed May 16 11:26:02 CEST 2018] Please check log file for more details: /root/.acme.sh/acme.sh.log
[Wed May 16 11:26:02 CEST 2018] The dns manual mode can not renew automatically, you must issue it again manually. You'd better use the other modes instead.
[Wed May 16 11:26:02 CEST 2018] Error renew storage.hijnn.net.

1 hour later:

Code:

[Wed May 16 12:26:04 CEST 2018] storage.hijnn.net:Verify error:Incorrect TXT record
[Wed May 16 12:26:04 CEST 2018] Please check log file for more details: /root/.acme.sh/acme.sh.log
[Wed May 16 12:26:05 CEST 2018] The dns manual mode can not renew automatically, you must issue it again manually. You'd better use the other modes instead.
[Wed May 16 12:26:05 CEST 2018] Error renew storage.hijnn.net.

And another hour later, repeated every hour since then:

Code:

[Wed May 16 13:26:02 CEST 2018] storage.hijnn.net:Challenge error: {"type":"urn:acme:error:malformed","detail":"Unable to update challenge :: The challenge is not pending.","status": 400}
[Wed May 16 13:26:02 CEST 2018] Please check log file for more details: /root/.acme.sh/acme.sh.log
[Wed May 16 13:26:03 CEST 2018] The dns manual mode can not renew automatically, you must issue it again manually. You'd better use the other modes instead.
[Wed May 16 13:26:03 CEST 2018] Error renew storage.hijnn.net.

To fix it I first have to issue a new cert: .acme.sh/acme.sh --issue --force -d storage.hijnn.net --dns --reloadcmd "/root/deploy_freenas.py"

Add the new value to the domain TXT record and then force a renew:
.acme.sh/acme.sh --renew --force -d storage.hijnn.net --dns --reloadcmd "/root/deploy_freenas.py"

But for some reason Freenas still uses the old certificate, even after restarting nginx. Although the certs have been changed:

Code:

3 -rw-r--r--  1 root  wheel  1647 May 17 22:35 ca.cer
4 -rw-r--r--  1 root  wheel  3806 May 17 22:35 fullchain.cer
3 -rw-r--r--  1 root  wheel  2159 May 17 22:35 storage.hijnn.net.cer
1 -rw-r--r--  1 root  wheel   509 May 17 22:35 storage.hijnn.net.conf
2 -rw-r--r--  1 root  wheel   985 May 17 22:34 storage.hijnn.net.csr
1 -rw-r--r--  1 root  wheel   212 May 17 22:34 storage.hijnn.net.csr.conf
3 -rw-r--r--  1 root  wheel  1679 Mar 11 10:24 storage.hijnn.net.key

I think I'll try one of the other renew methods, but I don't like the idea to connect my nas to the internet...

 

[jsherm101]

Has this been tested with 11.2? I'm hitting a bunch of errors in deploy_freenas.py when trying to import to the GUI:

Code:

[Thu May 17 22:37:21 PDT 2018] Run reload cmd: /root/deploy_freenas.py

Traceback (most recent call last):

  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn

	(self.host, self.port), self.timeout, **extra_kw)

  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection

	raise err

  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection

	sock.connect(sa)

ConnectionRefusedError: [Errno 61] Connection refused


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen

	chunked=chunked)

  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request

	conn.request(method, url, **httplib_request_kw)

  File "/usr/local/lib/python3.6/http/client.py", line 1239, in request

	self._send_request(method, url, body, headers, encode_chunked)

  File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request

	self.endheaders(body, encode_chunked=encode_chunked)

  File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders

	self._send_output(message_body, encode_chunked=encode_chunked)

  File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output

	self.send(msg)

  File "/usr/local/lib/python3.6/http/client.py", line 964, in send

	self.connect()

  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 166, in connect

	conn = self._new_conn()

  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn

	self, "Failed to establish a new connection: %s" % e)

urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x808084c18>: Failed to establish a new connection: [Errno 61] Connection refused


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 440, in send

	timeout=timeout

  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen

	_stacktrace=sys.exc_info()[2])

  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment

	raise MaxRetryError(_pool, url, error or ResponseError(cause))

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='nas.lanukah.com', port=80): Max retries exceeded with url: /api/v1.0/system/certificate/import/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x808084c18>: Failed to establish a new connection: [Errno 61] Connection refused',))


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/root/deploy_freenas.py", line 45, in <module>

	"cert_privatekey": priv_key,

  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 112, in post

	return request('post', url, data=data, json=json, **kwargs)

  File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 58, in request

	return session.request(method=method, url=url, **kwargs)

  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 508, in request

	resp = self.send(prep, **send_kwargs)

  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 618, in send

	r = adapter.send(request, **kwargs)

  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 508, in send

	raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='nas.lanukah.com', port=80): Max retries exceeded with url: /api/v1.0/system/certificate/import/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x808084c18>: Failed to establish a new connection: [Errno 61] Connection refused',))

[Thu May 17 22:37:24 PDT 2018] Reload error for :

 

[danb35]

Has this been tested with 11.2?

No; as 11.2 hasn't been released yet it isn't possible to test with that version. But the errors you're posting look like your server is refusing connections on port 80. Do you have HTTP enabled in the GUI, or only HTTPS?