Having this same issue myself - I think one complication is that there are multiple valid formats for RSA keys (and maybe certificates) and FreeNAS' validation may currently be too strict to handle them correctly.
I too generated an RSA key (2048 bit), CSR, and signed with my internal CA. Attempting to import the concatenated key+signed certificate into WebGUI fails with same message as OP. My private key is PEM encoded, as near as I can tell, and contains the following header/footer:
Code:
-----BEGIN PRIVATE KEY-----
[base64 encoded key data]
-----END PRIVATE KEY-----
The signed certificate is encoded with the following:
Code:
-----BEGIN CERTIFICATE-----
[base64 encoded cert data]
-----END CERTIFICATE-----
Resulting validation error on upload from SSL section on web gui:
RSA or DSA private key not found
I don't believe it's feasible or correct that simply putting the string "RSA" into the key header/footer is enough to do anything significant for validation and in fact, if the same encoded key data from the above format is simply left between modified header/footer this creates an invalid private key as seen here:
$ openssl rsa -in server.key -noout -text
unable to load Private Key
97306:error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:/SourceCache/OpenSSL098/OpenSSL098-47.2/src/crypto/asn1/tasn_dec.c:1315:
97306:error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:/SourceCache/OpenSSL098/OpenSSL098-47.2/src/crypto/asn1/tasn_dec.c:827:
97306:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:/SourceCache/OpenSSL098/OpenSSL098-47.2/src/crypto/asn1/tasn_dec.c:747:Field=n, Type=RSA
97306:error:0D09A00D:asn1 encoding routines:d2i_PrivateKey:ASN1 lib:/SourceCache/OpenSSL098/OpenSSL098-47.2/src/crypto/asn1/d2i_pr.c:99:
97306:error:0907B00D:PEM routines:PEM_READ_BIO_PRIVATEKEY:ASN1 lib:/SourceCache/OpenSSL098/OpenSSL098-47.2/src/crypto/pem/pem_pkey.c:125:
So leaving the encoded key data alone and simply modifying the header and footer gives this message when the invalid key is attempted to be parsed in web gui:
RSA or DSA private key is not valid
Now I do notice that when I run the following command (essentially telling openssl to modify what I think is the functional PEM formatted private key into its output representation of a PEM formatted private key:
$ openssl rsa -inform PEM -in server.key -outform PEM -out server.key.pem
...that the resulting key file now contains different base64 encoded data between the header and the footer and they are updated to what most people seem to expect:
Code:
-----BEGIN RSA PRIVATE KEY-----
[base64 encoded key data]
-----END RSA PRIVATE KEY-----
Concatenating this "new" PEM-formatted private key with the certificate and pasting that into the SSL upload field results in the message:
SSL successfully updated.
This too was a bit touchy for me; the first time I did it, it didn't seem to take because when I re-connected to the server, it was still using the iXsystems, Inc. self-signed cert. However the second time I did it (walking through to write this) it seems to have taken:
$ openssl s_client -connect nas.example.com:443
...
subject=/C=US/ST=AZ/L=Phoenix/O=Internal/OU=Home/CN=nas.example.com/name=EasyRSA/emailAddress=
root@example.com
issuer=/C=US/ST=AZ/L=Phoenix/O=Internal/OU=Home/CN=example.com CA/name=EasyRSA/emailAddress=
root@example.com
This can be recreated by generating a signing CA and server certificate using the easy-rsa utilities from OpenVPN (
https://github.com/OpenVPN/easy-rsa). It's simply a set of wrapper scripts around OpenSSL that creates signed RSA key/certificate pairs in PEM format but it introduces the valid but not-currently-handled-by-freenas-webgui key/certificate format that people seem to be running into. (Valid meaning like others have said that when these keys and certificates are used with other common network services, they function properly and do not run into any issues. This is notably the standard PKI toolkit distributed and used with the OpenVPN project). The output private key (.key) will be in the format that doesn't work/import (containing the -----BEGIN PRIVATE KEY----- header); the output certificate (.crt) is also in a format that won't work for simply concatenating with the key because it contains the readable text representation of the key data above the key, which has to be removed.
If this behavior could be nailed down and fixed I think it would solve the issue that folks have been reporting with the certificate importer for a while now.
One other thing: it's unclear to me what the purpose of the top set of fields are on the SSL settings page in the web gui; they look like parameters for creating a CSR or similar, but as the only field that seems to matter is the one that accepts the key+signed certificate PEM pair, they don't seem to serve a purpose. What am I missing?
Thanks!
DS