MacLemon
Cadet
- Joined
- Dec 14, 2017
- Messages
- 5
I need to migrate my existing encrypted zpool (RAIDZ2) onto a new, larger and also encrypted zpool. Luckily I do have a test system which I can test the whole procedure on and trash it as many times as I like.
I have read many threads on migration of pools but none of them actually takes into account encrypted pools. I haven't found actually working instructions regarding this in the documentation, this forum, with $ search engine on the interwebs either. (Links are provided at the bottom of this posting.)
Hardware details are also at the bottom of this posting. I don't consider the hardware even remotely being part of the problem. This is purely an issue of process or software.
The problem I'm running into is that I cannot import/remount an encrypted pool anymore, not with the geli keys, not with the geli_recovery keys.
Here's the procedure, based off of the thread Howto: migrate data from one pool to a bigger pool which deals with unencrypted source and destination pools only.
0. Scrubbing all the pools to make sure all my source data storage is fine.
Everything is OK.
1. I'm moving the system-dataset to the freenas-boot drive.
System > System Dataset > System Dataset pool: freenas-boot [V]
2. I'm taking a snapshot of the freenas-boot zpool just in case.
3. I'm creating a backup of my system config using the GUI.
System > General > [Save Config]
Export Password Secret Seed: [X]
[OK]
4. I'm creating a backup of the key files in /data/geli.
I copy those keys off of the FreeNAS box via SSH to my laptop to have backup.
5. I'm creating a backup of the keys via the GUI to my laptop.
Storage [Volumes]: Select zsource [Download key]
I make sure that I know which pool this key (file named geli.key) belongs to.
I [add recovery key] as well and save the geli_recovery.key to my laptop as well.
I do the same for the freshly created zdestination zpool as well and make sure that I know these keys are the GELI keys for the destination pool I want to migrate to.
6. I'm using the command line to create a snapshot.
I'm sending the data from zsource to zdestination.
Yes, this is deliberate overwriting the destination pool which is empty. (Created with the GUI with the Volume manager.)
--
So far, this has been simple and working fine.
Now the next steps would be to rename the zsource pool to old-source and the zdestination pool to become the new zsource.
7. Swap the pools by renaming them
Export both pools.
[cmdzpool export zdestination[/cmd
Reimport the old source pool and rename it to source-old.
zpool import zsource zsource-old
I'll leave it imported for the moment.
Reimport the new zdestination pool to become the new zsource.
zpool import zdestination zsource
Steps that I would do after I managed to successfully reimport the new, encrypted pool into FreeNAS:
8. Exporting both pools again.
The zsource-old pool will be removed, as in physically taking the disks out when everything has been done successfully.
9. Import the new pool via the GUI to make FreeNAS aware of it.
Storage > [Import Volume]
Select all 8 disks used up by the new pool
O decrypt Disks
Use the matching
Provide the correct passphrase
[Import]
10. Restore the configuration backup created in step 3.
System > General: [Upload Config]
This should restore my SMB and AFP Sharepoints to a working state.
Reboot the whole box and test if everything is ok.
11. Live happily ever after.
ZFS should be happy that the capacity for the pool is below 80% again.
I shall be happy that I've got a larger zpool now which I can use to store data.
---
What keeps me from doing this with my production setup yet?
I've never managed to reimport an encrypted pool in FreeNAS. Neither by using
I'm obviously making some mistake in the whole process, yet I've most likely overlooked it too many times now to being able to find it.
The problem I run into is somewhere with steps 7 to 9 (including).
Any help pointing my brain in the direction of my error is welcome. I can test all suggestions before actually touching any production data.
Thanks!
Best regards
MacLemon
---
Short Hardware overview
Test System
HP Proliant Microserver G8
CPU: intel G1610T, 16GB RAM, FreeNAS 11.1-RELEASE on USB stick, 2 HDDs for testing (HGST Deskstar NAS v2 6TB) (Nothing fancy, just a tiny test box.)
Production system
This is the box I actually want to migrate.
SOC: SUPERMICRO A1SAM-2750F
Memory: 32GB RAM (4 * 8GB Samsung)
HBA: IBM ServeRAID M1015, crossflashed to LSI 9211-8i in IT mode
FreeNAS: 11.1-RELEASE (stable train) on SATA-DOM
zpool (“zsource”): 6 wide RAIDZ2 HGST Deskstar NAS 4TB.
zpool (“zdestination”): 8 wide RAIDZ2 HGST Deskstar NAS v2 6TB
---
Links I've considered before posting this:
From the FreeNAS Documentation:
8.1.6.1 Importing an Encrypted Pool
8.1.8.1. Managing Encrypted Volumes
From this forum:
Howto: migrate data from one pool to a bigger pool
From the FreeNAS Bug Tracker:
https://forums.freenas.org/index.ph...a-gui-but-able-to-via-cli.59430/#post-420655 Unable to import encrypted pool via GUI, but able to via CLI
Suggests, that 11-RELEASE-U5 does have a bug with importing encrypted pools.
11.1-RC3 is said to be fixed already. (I'm running FreeNAS 11.1-RELEASE now.)
All the nitty gritty tech hardware details
An extensive description of the exact hardware used in my production system can be found at [Feedback sought] Enthusiastic NAS build [18 month usage update 2017] in the Serve the home forums.
I have read many threads on migration of pools but none of them actually takes into account encrypted pools. I haven't found actually working instructions regarding this in the documentation, this forum, with $ search engine on the interwebs either. (Links are provided at the bottom of this posting.)
Hardware details are also at the bottom of this posting. I don't consider the hardware even remotely being part of the problem. This is purely an issue of process or software.
The problem I'm running into is that I cannot import/remount an encrypted pool anymore, not with the geli keys, not with the geli_recovery keys.
Here's the procedure, based off of the thread Howto: migrate data from one pool to a bigger pool which deals with unencrypted source and destination pools only.
0. Scrubbing all the pools to make sure all my source data storage is fine.
Everything is OK.
1. I'm moving the system-dataset to the freenas-boot drive.
System > System Dataset > System Dataset pool: freenas-boot [V]
2. I'm taking a snapshot of the freenas-boot zpool just in case.
zfs snapshot -r freenas-boot@migration
3. I'm creating a backup of my system config using the GUI.
System > General > [Save Config]
Export Password Secret Seed: [X]
[OK]
4. I'm creating a backup of the key files in /data/geli.
cp -r /data/geli /data/geli_migration
I copy those keys off of the FreeNAS box via SSH to my laptop to have backup.
5. I'm creating a backup of the keys via the GUI to my laptop.
Storage [Volumes]: Select zsource [Download key]
I make sure that I know which pool this key (file named geli.key) belongs to.
I [add recovery key] as well and save the geli_recovery.key to my laptop as well.
I do the same for the freshly created zdestination zpool as well and make sure that I know these keys are the GELI keys for the destination pool I want to migrate to.
6. I'm using the command line to create a snapshot.
zfs snapshot -r zsource@migrate
I'm sending the data from zsource to zdestination.
zfs send -R zsource@migrate | pv | zfs receive -F zdestination
pv(1)
is only there to get progress indication so I know that data is transferring and how fast it's moving. There's no difference regarding the zfs send | zfs receive
without using pv(1)
to show progress.Yes, this is deliberate overwriting the destination pool which is empty. (Created with the GUI with the Volume manager.)
--
So far, this has been simple and working fine.
Now the next steps would be to rename the zsource pool to old-source and the zdestination pool to become the new zsource.
7. Swap the pools by renaming them
Export both pools.
zpool export zsource
[cmdzpool export zdestination[/cmd
Reimport the old source pool and rename it to source-old.
zpool import zsource zsource-old
I'll leave it imported for the moment.
Reimport the new zdestination pool to become the new zsource.
zpool import zdestination zsource
Steps that I would do after I managed to successfully reimport the new, encrypted pool into FreeNAS:
8. Exporting both pools again.
zpool export zsource
zpool export zsource-old
The zsource-old pool will be removed, as in physically taking the disks out when everything has been done successfully.
9. Import the new pool via the GUI to make FreeNAS aware of it.
Storage > [Import Volume]
Select all 8 disks used up by the new pool
O decrypt Disks
Use the matching
geli.key
Provide the correct passphrase
[Import]
10. Restore the configuration backup created in step 3.
System > General: [Upload Config]
This should restore my SMB and AFP Sharepoints to a working state.
Reboot the whole box and test if everything is ok.
11. Live happily ever after.
ZFS should be happy that the capacity for the pool is below 80% again.
I shall be happy that I've got a larger zpool now which I can use to store data.
---
What keeps me from doing this with my production setup yet?
I've never managed to reimport an encrypted pool in FreeNAS. Neither by using
geli.key
from the corresponding pool not with geli_recovery.key
. (This would result in a complete loss of data, which I obviously want to avoid.)I'm obviously making some mistake in the whole process, yet I've most likely overlooked it too many times now to being able to find it.
The problem I run into is somewhere with steps 7 to 9 (including).
Any help pointing my brain in the direction of my error is welcome. I can test all suggestions before actually touching any production data.
Thanks!
Best regards
MacLemon
---
Short Hardware overview
Test System
HP Proliant Microserver G8
CPU: intel G1610T, 16GB RAM, FreeNAS 11.1-RELEASE on USB stick, 2 HDDs for testing (HGST Deskstar NAS v2 6TB) (Nothing fancy, just a tiny test box.)
Production system
This is the box I actually want to migrate.
SOC: SUPERMICRO A1SAM-2750F
Memory: 32GB RAM (4 * 8GB Samsung)
HBA: IBM ServeRAID M1015, crossflashed to LSI 9211-8i in IT mode
FreeNAS: 11.1-RELEASE (stable train) on SATA-DOM
zpool (“zsource”): 6 wide RAIDZ2 HGST Deskstar NAS 4TB.
zpool (“zdestination”): 8 wide RAIDZ2 HGST Deskstar NAS v2 6TB
---
Links I've considered before posting this:
From the FreeNAS Documentation:
8.1.6.1 Importing an Encrypted Pool
8.1.8.1. Managing Encrypted Volumes
From this forum:
Howto: migrate data from one pool to a bigger pool
From the FreeNAS Bug Tracker:
https://forums.freenas.org/index.ph...a-gui-but-able-to-via-cli.59430/#post-420655 Unable to import encrypted pool via GUI, but able to via CLI
Suggests, that 11-RELEASE-U5 does have a bug with importing encrypted pools.
11.1-RC3 is said to be fixed already. (I'm running FreeNAS 11.1-RELEASE now.)
All the nitty gritty tech hardware details
An extensive description of the exact hardware used in my production system can be found at [Feedback sought] Enthusiastic NAS build [18 month usage update 2017] in the Serve the home forums.
Last edited by a moderator: