Hello all! I'm trying to store multiple XenServer VM disk images shared via NFS but need the data to be encrypted at rest. My threat model is to deal only with physical theft of the drives or server. Currently using Solaris for the job, but I have some hardware incompatibility. 25-disk storage server runs in a shared closet that I have limited access to. Unfortunately, FreeNAS' rekeying process scares me and the need to reboot the server after a disk replacement is not ideal. Also, native ZFS encryption isn't ready for prime-time in FreeNAS (or possibly even ported).
What I'm thinking of doing is running FreeNAS 11 on bare metal with access to the hardware HBA with unencrypted data. This should give me all the nice ZFS correctness guarantees about the data written to disk. Then, I plan to install FreeNAS as a Bhyve VM and then use FreeNAS' FDE in the VM, presenting the underlying pool-protected storage as a single drive. Since this VM will live on some ZVOLs, I believe this means that individual disk failures will be hidden from the virtualized FreeNAS but I can transparently (to the VM) hot-swap drives, managing the underlying pool from the bare-metal FreeNAS. Thus, I can avoid the rekeying process.
What I am concerned about is whether I will still get the correctness guarantees of ZFS if I run my storage in this fashion. I know ZFS requires the underlying storage to not "lie" about when and what data is written. I'm guessing I may also see a performance increase since the documentation states that performance may suffer when running FDE on systems with more than 8 disks. Since that doesn't refer to throughput of individual disks or the pool capacity, I'm guessing this has to do with the fact that disks are keyed differently. I do not need the additional protection afforded by FreeNAS re: disposing of disks; I am okay with running a magnet over non-functioning disks and then taking a sledgehammer to them.
So does a virtualized FreeNAS 11 with FDE on FreeNAS 11 bare-metal provide the same integrity guarantees as FreeNAS 11 on bare-metal? Must I avoid scrubbing from within the VM? Are there any settings I need to set? Anything else I should be aware of for this setup?
What I'm thinking of doing is running FreeNAS 11 on bare metal with access to the hardware HBA with unencrypted data. This should give me all the nice ZFS correctness guarantees about the data written to disk. Then, I plan to install FreeNAS as a Bhyve VM and then use FreeNAS' FDE in the VM, presenting the underlying pool-protected storage as a single drive. Since this VM will live on some ZVOLs, I believe this means that individual disk failures will be hidden from the virtualized FreeNAS but I can transparently (to the VM) hot-swap drives, managing the underlying pool from the bare-metal FreeNAS. Thus, I can avoid the rekeying process.
What I am concerned about is whether I will still get the correctness guarantees of ZFS if I run my storage in this fashion. I know ZFS requires the underlying storage to not "lie" about when and what data is written. I'm guessing I may also see a performance increase since the documentation states that performance may suffer when running FDE on systems with more than 8 disks. Since that doesn't refer to throughput of individual disks or the pool capacity, I'm guessing this has to do with the fact that disks are keyed differently. I do not need the additional protection afforded by FreeNAS re: disposing of disks; I am okay with running a magnet over non-functioning disks and then taking a sledgehammer to them.
So does a virtualized FreeNAS 11 with FDE on FreeNAS 11 bare-metal provide the same integrity guarantees as FreeNAS 11 on bare-metal? Must I avoid scrubbing from within the VM? Are there any settings I need to set? Anything else I should be aware of for this setup?