SOLVED FreeNAS-9.10.2-U1 // Unable to import encrypted pool from GUI

[HolyK]

Hi, this is a "spin-off" of the upgrade plan but as it is quite specific "issue" i am creating a separate thread.

FreeNAS version is quite old: FreeNAS-9.10.2-U1

Show : Full system specs

OS: FreeNAS-9.10.2-U1 (on USB dongle)
MB: SuperMicro MBD-X10SL7-F - Intel C222
CPU: Intel Core i3-4130
RAM: Kingston Value 16GB (2x8GB) DDR3 1333 ECC
PSU: Enermax ErPRO80+ 350W
Pool1: RAIDZ2 / 6x Western Digital WD Green WD20EZRX 2TB - called "MIRROR"
Pool2: Mirror / 2x WD Red (EFRX), 3,5" - 2TB
Case: Fractal Design DEFINE R4 Black Pearl

So i have two encrypted pools. One is RAIDZ2 (a big archive one) and second is mirrored one (2 disks). I will need to rename (export/import) the big one but before i do i am testing the steps on the smaller (mirrored) pool.
First of all YES i have a backup of the mirror pool i am playing with (did a zfs send/recv and sent everything to the first archive pool). The problem here is that i need to achieve the rename on the big one and i have no place to backup all of the data. So i need to know exactly how to proceed.

- In order to "rename" the pool it has to be exported, then imported with different (temp) name, then exported again and finally imported with a new name. Whole process witn UNencrypted pool is like:

Code:

zpool export mirror
zpool import mirror mirrornew
zpool export mirrornew
zpool import mirrornew

Now to the problem with the encrypted pools...
- I've locked the pool via GUI first
- Then i did a "DETACH" from the GUI without checking the "Destroy" and "Delete shares configuration". So it actually did a "zpool export -f mirror" (this is from zpool history mirror)
- Before doing any changes i wanted to import the pool again but apparently there is a GUI bug where the disks are NOT listed in "Import volume" (after selecting "encrypted volumes" in first step). I've tried to restart whole system but w/o change.

- Some info about the devices:

Show : camcontrol devlist

camcontrol devlist
<WDC WD20EFRX-68EUZN0 80.00A80> at scbus1 target 0 lun 0 (pass6,ada0)
<WDC WD20EFRX-68EUZN0 80.00A80> at scbus2 target 0 lun 0 (pass7,ada1)

Show : geom disk list

geom disk list
Geom name: ada0
Providers:
1. Name: ada0
Mediasize: 2000398934016 (1.8T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r0w0e0
descr: WDC WD20EFRX-68EUZN0
lunid: 5xxxxxxxxxxxxxxxxf
ident: WD-WMCxxxxxxxx0
rotationrate: 5400
fwsectors: 63
fwheads: 16

Geom name: ada1
Providers:
1. Name: ada1
Mediasize: 2000398934016 (1.8T)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r0w0e0
descr: WDC WD20EFRX-68EUZN0
lunid: 5xxxxxxxxxxxxxxxx8
ident: WD-WMCxxxxxxxx1
rotationrate: 5400
fwsectors: 63
fwheads: 16

Show : gpart show

gpart show
=> 34 3907029101 ada0 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834696 2 freebsd-zfs (1.8T)
3907029128 7 - free - (3.5K)

=> 34 3907029101 ada1 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834696 2 freebsd-zfs (1.8T)
3907029128 7 - free - (3.5K)

- So i went to cmd line and attached/decrypted the devices manually by:

Code:

geli attach -k /data/geli/c68blahblahblahblah.key /dev/ada0p2
geli attach -k /data/geli/c68blahblahblahblah.key /dev/ada1p2

Show : zpool import

zpool import
pool: mirror
id: 169XXXXXXXXXXXXXX91
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

mirror ONLINE
mirror-0 ONLINE
ada0p2.eli ONLINE
ada1p2.eli ONLINE

Code:

zpool import mirror

Code:

zpool list
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
mirror     1.81T  1.37T   455G         -    41%    75%  1.00x  ONLINE  -

- Then i've fixed the mount points to match the original one

Code:

zfs set mountpoint=/mnt/mirror mirror
zfs mount -a

Code:

geli status
      Name  Status  Components
ada0p2.eli  ACTIVE  ada0p2
ada1p2.eli  ACTIVE  ada1p2

Code:

glabel status
                                      Name  Status  Components
gptid/0f6cxxxxx-xxxxx-xxx-xxxxx-xxxxxxxxxxxxa     N/A  ada0p1
gptid/0fe5xxxxx-xxxxx-xxx-xxxxx-xxxxxxxxxxxxa     N/A  ada1p1

- So at this point i can access the data but the GUI is un-aware about it! The pool is not listed under "View volumes" and more over it provides the disks in Device manager (dangerous!).
- I've checked content of storage_encrypteddisk table in freenas-v1.db and the records are not there.

This is obvious as I've imported the pools from CLI instead of the GUI ...

So the question is HOW can I import the pool via GUI properly if the import volume does not show the disks? (I know i could just restore my config backup and restart but that is not what i want to do...).

I know the version of FreeNAS is old but it is actually a preparation for migration to most recent one so any help/comments would be appreciated :]

Thanks!

Alex

 

[HolyK]

Ok so here is a fix (in case someone faces the same issue). Note that you're doing bellow on your own. I am not taking any responsibilities you you nuke your data. You should understand what the commands bellow are doing and you need to know your environment (like which disk belongs to which pool, their serial#, pool names, etc...).

So here we go...

- Copy the encryption key from your backup (you have it, right?) back to the /data/geli/ (because it gets deleted upon volume detach !)

- Get the necessary data from your config backup (you have that one as well i hope)

Code:

sqlite3 /path/to/cfgbackup.db
select * from storage_disk;
select * from storage_encrypteddisk;
select * from storage_mountpoint;
select * from storage_volume;

- Now check the same in your "live" DB

Code:

sqlite3 /data/freenas-v1.db
select * from storage_disk;
select * from storage_encrypteddisk;
select * from storage_mountpoint;
select * from storage_volume;

- You will see missing lines in storage_encrypteddisk and storage_volume tables. The other two should be same.

- You have to unlock the pool manually via CLI first. Get the gptid of the devices of the missing pool. Easy source of that is the storage_encrypteddisk table from the BACKUP config, third column. Cross-check that with device serial number(s)!

Code:

geli attach -k /data/geli/encrkey.key /dev/gptid/0faaxxxxxx-xxxxx-xxx-xxxx-xxxxxxxxxxxxx
geli attach -k /data/geli/encrkey.key /dev/gptid/0fbbxxxxxx-xxxxx-xxx-xxxx-xxxxxxxxxxxxx

//Do the same for all devices which belongs to the encrypted pool (It will prompt for passphrase)

- Now import the pool from CLI

Code:

zpool import
zpool import mypoolname

- Fix the mount points (if you have to)

Code:

zfs set mountpoint=/mnt/mypoolname mypoolname
zfs mount -a

- Verify that the pool is healthy (zpool status) and you can access the data. If not then stop right there and do not continue.

- If the pool is unlocked and data accessible now you can fix the DB records so GUI is aware about the pool.

- Open the DB and add the missing lines - REPLACE the values by the ones you got from your backup CFG!

Code:

INSERT INTO storage_encrypteddisk (encrypted_volume_id,id,encrypted_provider,encrypted_disk_id) VALUES(2,1,'gptid/0faaxxxxxx-xxxxx-xxx-xxxx-xxxxxxxxxxxxx','{serial}WD-WMCXXXXXXXXX');
INSERT INTO storage_encrypteddisk (encrypted_volume_id,id,encrypted_provider,encrypted_disk_id) VALUES(2,2,'gptid/0fbbxxxxxx-xxxxx-xxx-xxxx-xxxxxxxxxxxxx','{serial}WD-WMCXXXXXXXXX');
//Add more similar lines if you have additional disks for that pool

INSERT INTO storage_volume (vol_guid,vol_name,vol_encryptkey,vol_fstype,vol_encrypt,id) VALUES ('xxxxxxxxxxxxxxxxxxxxxx','mypoolname','c77axxxxxxx-xxxxx-xxx-xxxx-xxxxxxxxxxxxx','ZFS',2,2);

- Re-check the records in the DB. It has to match the records from the backup CFG

Code:

sqlite3 /data/freenas-v1.db
select * from storage_encrypteddisk;
select * from storage_volume;

- Now just REBOOT the server. After restart you will get an error that some devices failed to decrypt or missing and your pool is unavailable. Zpool status shows "UNAVAIL". GUI shows the volume as "Unknown". Just go to CLI again and use zpool export mypoolname. After that you should see the pool in GUI as "LOCKED". Hit the Unlock button, add the passphrase and confirm. It will unlock and import the pool properly. Data are intact and zpool status shows ONLINE.

Done :]