How Do I Decrypt my ZFS drive via SSH?

Status
Not open for further replies.

Cosmo_Kramer

Contributor
Joined
Jan 9, 2013
Messages
103
Hello,
I'd like to know how to decrypt my ZFS drive via SSH in the case that the power goes out for a long time (And the NAS powers off/on once the UPS dies) while I'm away.

I would port forward the webGUI but I use a Surface RT when I'm away and it seems IE11 on here cannot process the okay button on the decrypt screen.

Could someone please help?

Thanks for your time.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Um, you'll have bigger problems if you try to do that because the FreeNAS WebGUI doesn't know when you go around it and do stuff in the background. Doing and end-around on the WebGUI can cause issues ranging from crashing FreeNAS to losing data.

The short advice is "Don't do this".
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
There's a perfectly "legal" way to do this from command line that will properly notify/sync the GUI: the new API. This small piece of python code will unlock a volume:

[PRE]import json
import requests
r = requests.post(
'https://localhost/api/v1.0/storage/volume/<volume_name>/unlock/',
auth=('root', '<root_password>'),
headers={'Content-Type': 'application/json'},
verify=False,
data=json.dumps({'passphrase': '<volume_passphrase>'})
)
print r.text[/PRE]
Use either http or https depending on how is your GUI configured. You can also wrap this into a small script that substitutes command line arguments for the variables instead of hard coding them.
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
1. I'm not sure that this will work with his surface.
Of course it will. The code above is supposed to be run inside a SSH session (notice the localhost). There are many options: he can just start python and paste the code; he can scp the a file over and run it; or he can keep it somewhere on FreeNAS and just run it.
2. This could change in the next release and he could end up in sh*t creek without a paddle.

Trying to do one-off stuff is for those that are advanced enough to figure it out for themselves.
You are now just spreading FUD. This is the official external API provided by FreeNAS to allow interaction with the system via a well defined, fully documented, interface. This is the answer to people not understanding the internals but messing with the config DB, running zfs/zpool from CLI and breaking stuff as the result.
The API gives you the option to do "GUI operations" remotely (or even locally from a script) properly -- the code that will run is the same that runs when you initiate the action via GUI -- this ensures that all the internals are in sync and consistent. You can use the API to create/manipulate volumes (including unlocking encrypted volumes), create new shares, enable/disable services, add/remove users, schedule/run scrubs, replace disks, ...
Also notice that the API is versioned (the v1.0 in the URL), so you can't accidentally run a changed API call with old parameters.
Please do not try to unnecessarily scare users away from this new (9.2.0) cool feature. It was added exactly for such use cases.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Ok, I'm not familiar with the API on FreeNAS. I'm not sure I've even seen the documentation on it before you linked it just now. I did print out the manual for 9.2.0, but I haven't had a chance to read it.

But I don't think its fair to claim I'm spreading FUD. I specifically remember someone who did some command line script with 8.3.1 and it all went to hell and a handbasket when 9.x came out. To boot, he did something wrong and trashed his pool while trying to fix his script.

So I might not be too familiar with this new API thing(and I'll definitely agree with that). But spreading FUD... far from it. You weren't here a year ago when lots of people were trying to do custom scripts on bootup for various things and with each FreeNAS update it not only broke their scripts, but in some cases cost them their data because they really didn't know what they were doing and just started trying things hoping to get it to work.

I'm not sure how "solid" the API is at this point as it really hasn't been tested by much of anyone I know of. Personally, I see it as a "work in progress" until its been tested a little more thoroughly.
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
But I don't think its fair to claim I'm spreading FUD.
I'm sorry about that comment, I though I removed it before hitting Post.
You weren't here a year ago when lots of people were trying to do custom scripts on bootup for various things and with each FreeNAS update it not only broke their scripts, but in some cases cost them their data because they really didn't know what they were doing and just started trying things hoping to get it to work.
I spent enough years in software maintenance to know what are users capable to doing to their systems/data ;).
When I saw the OP's question I fired up an VM to see how difficult it is. I unlocked the pool (config DB, keys in /data/geli, geli attach, zpool import, notifier.py, ...), I can do that properly, no problem. However, I decided to not provide the instructions as they were a bit complex even for simple cases (only one pool, no old keys in /data/geli) and much more complex if I wanted to cover all cases. Then I remembered the new API, checked the documentation and figured you only need one HTTP request (one line in python) to unlock the pool. Btw., the API is not python specific. I used python as it is available in FreeNAS (and the documentation uses it as well), but anything that is able to generate HTTP requests will work.
I'm not sure how "solid" the API is at this point as it really hasn't been tested by much of anyone I know of.
Time to increment that counter :).
There is nothing to be afraid of, there aren't hundreds of new untested lines of code. This is what runs when you use the unlock API:
Code:
def unlock(self, request, **kwargs):
    self.method_check(request, allowed=['post'])

    bundle, obj = self._get_parent(request, kwargs)

    deserialized = self.deserialize(
        request,
        request.body,
        format=request.META.get('CONTENT_TYPE', 'application/json'),
    )
    form = UnlockPassphraseForm(
        data=deserialized,
    )
    if not form.is_valid():
        raise ImmediateHttpResponse(
            response=self.error_response(request, form.errors)
        )
    else:
        form.done(obj)
    return HttpResponse('Volume has been unlocked.', status=202)

On line 6 it decodes the parameters you sent in the request. On line 11 it creates the Unlock form (UnlockPassphraseForm()) and fills in the form fields based on the received data. It is exactly the same form you use when you unlock the pool via GUI, just this one isn't displayed anywhere. On line 14 it runs the form validation (form.is_valid()) -- again, the same validation that happens when you use the form in GUI. Any problem is reported immediately. On line 19 it "clicks" the OK button on the form (form.done()). And that's it, the volume is unlocked.
Personally, I see it as a "work in progress" until its been tested a little more thoroughly.
You never tried it, but you see it as a "work in progress", hmm...
The great thing about APIs is that you can often easily automate the testing. There actually is a nice test suite that verifies the FreeNAS API functionality: https://github.com/freenas/freenas/tree/master/gui/api/tests
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Yep. I've never tried it, but I see it as a "work in progress". Not because I haven't tested it but because I haven't seen anyone really use it extensively yet. I think this is only the second thread I've seen discussing the FreeNAS API(the first one asking when/if one would be implemented). If lots of people were using it I'd feel safer about it. But as of now there hasn't been many people testing it.


Just look at how many tickets I've created involving the encryption system. There's been many potential data-killing bugs in the past. One person lost their pool in the last week or so from one in 9.2.0(I made a ticket for that one too). Part of the problem is that so few people use it. I'm using it only because I wanted to see how good it was. I have backups of the most important stuff on the encrypted pool and I chose to encrypt it as nobody else really uses it, so its often difficult to claim it really does work well.
 

George51

Contributor
Joined
Feb 4, 2014
Messages
126
There's a perfectly "legal" way to do this from command line that will properly notify/sync the GUI: the new API. This small piece of python code will unlock a volume:

[PRE]import json
import requests
r = requests.post(
'https://localhost/api/v1.0/storage/volume/<volume_name>/unlock/',
auth=('root', '<root_password>'),
headers={'Content-Type': 'application/json'},
verify=False,
data=json.dumps({'passphrase': '<volume_passphrase>'})
)
print r.text[/PRE]
Use either http or https depending on how is your GUI configured. You can also wrap this into a small script that substitutes command line arguments for the variables instead of hard coding them.

Of course it will. The code above is supposed to be run inside a SSH session (notice the localhost). There are many options: he can just start python and paste the code; he can scp the a file over and run it; or he can keep it somewhere on FreeNAS and just run it.

Evening - new user here so please forgive any ignorance.

What I am looking for is the ability to mount a encypted pool by windows task scheduler from a computer over LAN.

Talk to me as if i am a complete idiot - i've grasped the very basics of FreeNas, currently I have one zpool of four harddrives, I am looking at adding more and encypting the lot. I do not require remote acces - doing all from a computer connect on LAN is fine.

Looking at this thread: http://forums.freenas.org/threads/u...s-volume-via-shell-or-remotely-via-ssh.12018/

Seems to be a different way to watch you suggested, and they import the pool afterwards would this be required?


What would be the steps required to take your small python code and use it via a computer on LAN to unlock a the pool so i could access it. Do i need to scp? the file over as you suggested? as i am under the impressioin that is just for accessing freenas remotely? any help is appreciated.

Kind Regards
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
Talk to me as if i am a complete idiot - i've grasped the very basics of FreeNas, currently I have one zpool of four harddrives, I am looking at adding more and encypting the lot. I do not require remote acces - doing all from a computer connect on LAN is fine.
I hope you are aware that you can't encrypt an existing pool. You need to create a new encrypted pool.
Looking at this thread: http://forums.freenas.org/threads/u...s-volume-via-shell-or-remotely-via-ssh.12018/

Seems to be a different way to watch you suggested, and they import the pool afterwards would this be required?
That code is from the pre-API days. It does just the absolute minimum to unlock the pool to be able to run rsync. However, the GUI is not aware that the pool was imported, the services are not restarted (CIFS), ...
If you unlock the pool via the API you get the same functionality as when you click on the Unlock button in GUI.
What would be the steps required to take your small python code and use it via a computer on LAN to unlock a the pool so i could access it. Do i need to scp? the file over as you suggested? as i am under the impressioin that is just for accessing freenas remotely? any help is appreciated.
You need to install python on the other machine. Modify the bold parts in the script to match your installation and run it.
 

George51

Contributor
Joined
Feb 4, 2014
Messages
126
I hope you are aware that you can't encrypt an existing pool. You need to create a new encrypted pool.

That code is from the pre-API days. It does just the absolute minimum to unlock the pool to be able to run rsync. However, the GUI is not aware that the pool was imported, the services are not restarted (CIFS), ...
If you unlock the pool via the API you get the same functionality as when you click on the Unlock button in GUI.

You need to install python on the other machine. Modify the bold parts in the script to match your installation and run it.
I am aware - I am adding more drives and changing my RAID type so that is okay with me.

Okay so the API is the way to go as that is exactly what I want. Okay so again sorry about the ignorance - steps i would take

1) install python on the computer which is connected over LAN
2) modify the code to my relevant details
3) use windows task scheduler to run the code whenever I wanted to unlock the pool (same effect as unlocking via GUI)

Do I have to enable/change any settings from default in the GUI before this will work?

Thank you for your help so far.

Regards
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
1) install python on the computer which is connected over LAN
2) modify the code to my relevant details
3) use windows task scheduler to run the code whenever I wanted to unlock the pool (same effect as unlocking via GUI)

Do I have to enable/change any settings from default in the GUI before this will work?
Yes, your steps are correct and you do not need to modify anything in FreeNAS.
 

George51

Contributor
Joined
Feb 4, 2014
Messages
126
Sorry but one more issue - I've set it all up now - and I can unlock the pool form the computer using the script - however the Volume always says its unavailable on windows unless I login to the GUI and refresh the share that its associated with Any clues as to why or how to fix it?

Cheers

Edit: Its like the share isn't starting - although it says its enabled. Also probably irrelavant but I've dropped the 'print r.text' off your script as the new version of python doesn't accept it as it is and it seemed to work without it.
 

George51

Contributor
Joined
Feb 4, 2014
Messages
126
Sorry but one more issue - I've set it all up now - and I can unlock the pool form the computer using the script - however the Volume always says its unavailable on windows unless I login to the GUI and refresh the share that its associated with Any clues as to why or how to fix it?

Cheers

Edit: Its like the share isn't starting - although it says its enabled. Also probably irrelavant but I've dropped the 'print r.text' off your script as the new version of python doesn't accept it as it is and it seemed to work without it.

Following on from this - I also upgraded Freenass at the same time - read reports about that causing errors with CIFS shares - so I reinstalled 9,2.0 with a fresh installation.

This has not solved the problem - the script you gave me unlocks the pool, and when i click on the icon to access the files within my windows computer It says 'An error occured .... ... local device name is already in use.... this connection has not been restored'

If i then go into the GUI, go into shares, and click edit on the CIFS share, click ok immediately - i can then access it from windows. Any suggestions on how to fix the script? Cheers
 

seeN

Cadet
Joined
Mar 12, 2014
Messages
7
This has not solved the problem - the script you gave me unlocks the pool, and when i click on the icon to access the files within my windows computer It says 'An error occured .... ... local device name is already in use.... this connection has not been restored'

If i then go into the GUI, go into shares, and click edit on the CIFS share, click ok immediately - i can then access it from windows. Any suggestions on how to fix the script? Cheers
perhaps you have to specify in the script that you want your cifs share to be enabled like the check mark in the gui?

[...]
Code:
def unlock(self, request, **kwargs):
    self.method_check(request, allowed=['post'])
 
    bundle, obj = self._get_parent(request, kwargs)
 
    deserialized = self.deserialize(
        request,
        request.body,
        format=request.META.get('CONTENT_TYPE', 'application/json'),
    )
    form = UnlockPassphraseForm(
        data=deserialized,
    )
    if not form.is_valid():
        raise ImmediateHttpResponse(
            response=self.error_response(request, form.errors)
        )
    else:
        form.done(obj)
    return HttpResponse('Volume has been unlocked.', status=202)

[...]. On line 11 it creates the Unlock form (UnlockPassphraseForm()) and fills in the form fields based on the received data. It is exactly the same form you use when you unlock the pool via GUI, just this one isn't displayed anywhere. [...]

Code:
data=json.dumps({'passphrase': '<volume_passphrase>'})

^^there has to follow something?

Since i try to make a decrypt script that enables cifs and the plugins jail automatically i will let you know when im successful.
 

seeN

Cadet
Joined
Mar 12, 2014
Messages
7
Code:
#!/usr/bin/env python
 
import json
import requests
 
r = requests.put(
 
        'http://<yournas>/api/v1.0/services/services/cifs/',
 
        auth=('root', '<yourpass>'),
 
        headers={'Content-Type': 'application/json'},
 
        verify=False,
 
        data=json.dumps({'srv_enable': True}),
 
)
 
print(r.text)

edit <words> - if your smb is disabled then this otherwise disable with False then this.
 
Status
Not open for further replies.
Top