Hello everyone,
this is my first post in this forum. Since it is not directly related to FreeNAS but to NAS, ZFS, and HBAs I decided to post it under "Off-Topic". I plan to upgrade my one year old NAS in terms of storage capacity. I did a lot of research in different forums for the last two weeks and know what I want to do, but I am still not sure about every detail and hope to get some advice here.
My current setup:
OS : Ubuntu Server 16.04.2 with ZFS on Linux
Mainboard : Supermicro X11SAE-F
SSD for OS : Intel SSD 600p 256 GB (PCIe x4 via M.2 Interface)
Processor : Intel Xeon E3-1270 v5 (max. 64 GB DDR4)
Memory : 16 GB Kingston ValueRAM KVR21E15D8/16 (DDR4 2133 MHz with ECC)
Power Supply : Seasonic PRIME 650 W Titanium
Case : IPC 3U-3416 (16-bay rackmount case)
HDD : 8x 2TB (i.e. case is filled to 50%)
Cable : 2x (SFF-8087 to 4x SATA)
ZFS : 1 zpool with 1 vdev with 8 HDDs in RAIDZ3 (zpool created with "/dev/disk/by-id/", not sda,sdb,...)
Use case : 2-3 users; Gb ethernet; store backups and share files; IOPS are not that important; data integrity and robustness are important
My idea is to get 8x 4TB HDDs, WD Red and Seagate IronWolf should be ok (?), but are actually specified only for up to 8-bay cases because of more vibrations in larger systems. I will get half of each, to avoid data loss due to HDDs with the same manufacturing defects (may not do much but also doesn't cost much). I plan to add them as second RAIDZ3 vdev to my existing zpool with an HBA. I got several questions on how to do this.
1) Would you buy a 9305-16i (16x SAS3 and PCIe3) or a 9201-16i (16x SAS2 and PCIe2) card? Or just an 8-port HBA and continue to use the 8 SATA connectors on the mainboard?
What I know: The 9201-16i should be more than enough in terms of bandwidth to connect 16 SATA3 HDDs, so there is no need to get SAS3 and PCIe3. But what about compatability and reuse in the future?
2) Regarding the mainboard SATA connectors: Would you recommend to continue using them?
My current opinion: Better free the SATA connectors to connect 2 SSD drives (if not now, maybe in future), which could be used as a separate zpool (mirror) for fast scratch space and temporary stuff. Regarding SLOG and L2ARC, I don't think I need them and if it should be better to install another 16 GB of RAM.
3) What would be the best way to go through the upgrade procedure?
My idea: I fill the currently empty bays with the 8 new drives, insert the HBA and connect both with the appropriate cables. When I power up the system now, what can I expect to happen? Are there drivers for the HBAs available in Ubuntu so I can immediately use the 8 new drives? Or do I have to manually install a driver or even flash a fitting firmware update on the HBA? Assuming the HBA is working and I can use the new drives, I would create a new zpool with a RAIDZ3 vdev using all the new drives. Then I would copy over the data on my current zpool (as a side effect this removes fragmentation). After this I would connect the old drives to the HBA and add them as a second vdev (RAIDZ3) to the new pool.
4) As I wrote above, I created the zpool with the disk-ids because I read it is not guaranteed that sda,sdb,... are the same between reboots. Does this also mean I could just connect the HDDs of my current zpool with the HBA and its working fine, assuming the HBA driver and firmware are set up?
I know that are a lot of questions, but I want to make sure I know what I do when upgrading my NAS. I have 2 backup copies of my data in case I accidentially delete the data that is currently on the NAS, but I would like to avoid this ;)
Thank you for your help!
this is my first post in this forum. Since it is not directly related to FreeNAS but to NAS, ZFS, and HBAs I decided to post it under "Off-Topic". I plan to upgrade my one year old NAS in terms of storage capacity. I did a lot of research in different forums for the last two weeks and know what I want to do, but I am still not sure about every detail and hope to get some advice here.
My current setup:
OS : Ubuntu Server 16.04.2 with ZFS on Linux
Mainboard : Supermicro X11SAE-F
SSD for OS : Intel SSD 600p 256 GB (PCIe x4 via M.2 Interface)
Processor : Intel Xeon E3-1270 v5 (max. 64 GB DDR4)
Memory : 16 GB Kingston ValueRAM KVR21E15D8/16 (DDR4 2133 MHz with ECC)
Power Supply : Seasonic PRIME 650 W Titanium
Case : IPC 3U-3416 (16-bay rackmount case)
HDD : 8x 2TB (i.e. case is filled to 50%)
Cable : 2x (SFF-8087 to 4x SATA)
ZFS : 1 zpool with 1 vdev with 8 HDDs in RAIDZ3 (zpool created with "/dev/disk/by-id/", not sda,sdb,...)
Use case : 2-3 users; Gb ethernet; store backups and share files; IOPS are not that important; data integrity and robustness are important
My idea is to get 8x 4TB HDDs, WD Red and Seagate IronWolf should be ok (?), but are actually specified only for up to 8-bay cases because of more vibrations in larger systems. I will get half of each, to avoid data loss due to HDDs with the same manufacturing defects (may not do much but also doesn't cost much). I plan to add them as second RAIDZ3 vdev to my existing zpool with an HBA. I got several questions on how to do this.
1) Would you buy a 9305-16i (16x SAS3 and PCIe3) or a 9201-16i (16x SAS2 and PCIe2) card? Or just an 8-port HBA and continue to use the 8 SATA connectors on the mainboard?
What I know: The 9201-16i should be more than enough in terms of bandwidth to connect 16 SATA3 HDDs, so there is no need to get SAS3 and PCIe3. But what about compatability and reuse in the future?
2) Regarding the mainboard SATA connectors: Would you recommend to continue using them?
My current opinion: Better free the SATA connectors to connect 2 SSD drives (if not now, maybe in future), which could be used as a separate zpool (mirror) for fast scratch space and temporary stuff. Regarding SLOG and L2ARC, I don't think I need them and if it should be better to install another 16 GB of RAM.
3) What would be the best way to go through the upgrade procedure?
My idea: I fill the currently empty bays with the 8 new drives, insert the HBA and connect both with the appropriate cables. When I power up the system now, what can I expect to happen? Are there drivers for the HBAs available in Ubuntu so I can immediately use the 8 new drives? Or do I have to manually install a driver or even flash a fitting firmware update on the HBA? Assuming the HBA is working and I can use the new drives, I would create a new zpool with a RAIDZ3 vdev using all the new drives. Then I would copy over the data on my current zpool (as a side effect this removes fragmentation). After this I would connect the old drives to the HBA and add them as a second vdev (RAIDZ3) to the new pool.
4) As I wrote above, I created the zpool with the disk-ids because I read it is not guaranteed that sda,sdb,... are the same between reboots. Does this also mean I could just connect the HDDs of my current zpool with the HBA and its working fine, assuming the HBA driver and firmware are set up?
I know that are a lot of questions, but I want to make sure I know what I do when upgrading my NAS. I have 2 backup copies of my data in case I accidentially delete the data that is currently on the NAS, but I would like to avoid this ;)
Thank you for your help!
Last edited:
