Hello folks,
this post is simply some kind of report or feedback about my experience with upgrading my 6x 4TB WD drives based zpool on an esxi based virtualized freenas system. This is not intended to be used as an general how-to, but it might help a minority of freenas users who might run into similar problems, especially with VMware or virtualization.
First of all my setup:
Project:
Procedure:
I will skip the part of upgrading esxi to v.6.5 because it is well documented in the WWW.
Since I wanted to do an "live upgrade" from an z1 to z2-pool with more storage, I needed to connect 6x WD Green + 7x Seagate drives simultaneously. My motherboard luckily provided the exact amount of ports for the new setup:
The first challenge, was to passthrough my internal SATA-Controller to freenas via esxi. Initially my controller (Intel Coporation Lynx Point AHCI Controller) was not detected by esxi. So first workaround:
SSH to esxi and enable device manually for passthrough.
Reboot and afterwards I was able to passthrough all my SATA devices to my virtualized freenas. But ooops, my SSD with data-storage was connected to the sata-port as well, so how to deal with that? Hm... find an USB-to-SATA-Adapter connect the SSD drive via USB and follow a guide to enable vmware esxi to use an USB-device as data-storage (http://www.virten.net/2015/10/usb-devices-as-vmfs-datastore-in-vsphere-esxi-6-0/). Reboot again and voila, data-storage via USB is working and vmware is able to passthrough my internal sata-controller to freenas.
After that I roughly followed depasseg's guide "How to migrate data from one pool to a bigger pool" -> https://forums.freenas.org/index.ph...te-data-from-one-pool-to-a-bigger-pool.40519/) except for the renaming because moving the system data-set to a different zpool was not possible due to the simple fact that I had no spare hdd/sdd left anymore. Luckily I didn't expose to many Shares, so renaming the path to the new folder was easy for all the shares. (Permissions have been copied thx to zfs snapshot and migration ability)
End of the story: Remove the old zpool, shutdown freenas, shutdown esxi. Remove the old drives, reconnect the new drives to the LSI controller via SAS-Cable, reconnect SSD via SATA, remove USB-data-storage hook, remove SATA-Controller passthrough, reboot, start esxi and finally freenas vm again. And last but not least be happy, that everything is up and running again! ;)
So if you should challenge something similar in the future, don't panic! It is definitely doable and will work out with a little bit of effort! So for all the conservatives, yes I'm still happy with my virtualized freenas!
thx and greetings
SebbaG
this post is simply some kind of report or feedback about my experience with upgrading my 6x 4TB WD drives based zpool on an esxi based virtualized freenas system. This is not intended to be used as an general how-to, but it might help a minority of freenas users who might run into similar problems, especially with VMware or virtualization.
First of all my setup:
- Motherboard: Asrock E3C224D4I-14S
- RAID-Controller: build in LSI 2308 flashed to IT-Mode and 'passthroughed' via vt-d to freenas
- Memory: 4x 8GB Crucial DDR3-1600 ECC DIMM CL11 Single = 32GB
- CPU: Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
- HDDs: 6x 4TB WD Green in one encrypted z1-pool
- System: esxi v.5.5 with 1 samsung 120GB SSD as storage pool
- VM1: virtualized FreeNAS-9.3-STABLE-201412090314 on esxi running with 24GB Memory
Project:
- Update: VMWare esxi v5.5 to v.6.5
- Replace 6x 4TB WD Green Z1-pool with 7x 8TB Seagate IronWolf Z2-pool
Procedure:
I will skip the part of upgrading esxi to v.6.5 because it is well documented in the WWW.
Since I wanted to do an "live upgrade" from an z1 to z2-pool with more storage, I needed to connect 6x WD Green + 7x Seagate drives simultaneously. My motherboard luckily provided the exact amount of ports for the new setup:
- SATA Controller: Intel® C224 : 4 x SATA3 6.0 Gb/s (from mini SAS connector), 2 x SATA2 3.0 Gb/s, Support RAID 0, 1, 5, 10 and Intel® Rapid Storage
- SAS Controller: LSI 2308: 8 x SAS2 6Gbps (from 2x mini SAS 8087 connector)
The first challenge, was to passthrough my internal SATA-Controller to freenas via esxi. Initially my controller (Intel Coporation Lynx Point AHCI Controller) was not detected by esxi. So first workaround:
SSH to esxi and enable device manually for passthrough.
vi /etc/passthru.mapCode:
[...] # INTEL AHCI 8086 8d62 d3d0 false
Reboot and afterwards I was able to passthrough all my SATA devices to my virtualized freenas. But ooops, my SSD with data-storage was connected to the sata-port as well, so how to deal with that? Hm... find an USB-to-SATA-Adapter connect the SSD drive via USB and follow a guide to enable vmware esxi to use an USB-device as data-storage (http://www.virten.net/2015/10/usb-devices-as-vmfs-datastore-in-vsphere-esxi-6-0/). Reboot again and voila, data-storage via USB is working and vmware is able to passthrough my internal sata-controller to freenas.
After that I roughly followed depasseg's guide "How to migrate data from one pool to a bigger pool" -> https://forums.freenas.org/index.ph...te-data-from-one-pool-to-a-bigger-pool.40519/) except for the renaming because moving the system data-set to a different zpool was not possible due to the simple fact that I had no spare hdd/sdd left anymore. Luckily I didn't expose to many Shares, so renaming the path to the new folder was easy for all the shares. (Permissions have been copied thx to zfs snapshot and migration ability)
End of the story: Remove the old zpool, shutdown freenas, shutdown esxi. Remove the old drives, reconnect the new drives to the LSI controller via SAS-Cable, reconnect SSD via SATA, remove USB-data-storage hook, remove SATA-Controller passthrough, reboot, start esxi and finally freenas vm again. And last but not least be happy, that everything is up and running again! ;)
So if you should challenge something similar in the future, don't panic! It is definitely doable and will work out with a little bit of effort! So for all the conservatives, yes I'm still happy with my virtualized freenas!
thx and greetings
SebbaG
