Hello everyone,
I am thinking about switching to TrueNAS and ZFS permanently. I have installed it twice to test a few things including upgrading HBA firmware.
In the following setup with 8+8+8=24 HDD I would like to hear if what I am planning to do is a smart choice.
VDEV 1:
8 * 1 TB HDD in RAIDZ-3
VDEV 2:
8 * 2 TB HDD in RAIDZ-3
VDEV 3:
8 * 3 TB HDD in RAIDZ-3
In a scenario where disk in VDEV 3 dies I would replace it with a 4 TB HDD. If all 8 disks in VDEV 3 are replaced by 4 TB HDDs I will have 33% additional storage capacity going from 3 to 4 TB per HDD. In other words, a 3 TB HDD gets thrown out and is directly replaced by a 4 TB HDD.
If a disk from VDEV 2 dies, I would remove it from the system and buy a 4 TB HDD. I would then proceed by unplugging a 3 TB HDD from VDEV 3 and then reformat it on another computer. Then I will use that 3 TB HDD and replace the failed disk in VDEV 2. Then I will add the newly bought 4 TB HDD and add it to VDEV 3. In other words a 2 TB HDD gets thrown out (from VDEV 2) and a 3 TB HDD (from VDEV 3) takes over its spot. The 3 TB HDD is taken over by the newly bought 4 TB HDD (new HDD added to VDEV 3).
If a disk from VDEV 1 dies, the same happens as with VDEV 2, but just with one more step. A disk from VDEV 2 will be wiped and made to fill the void in VDEV 1. Then the void from VDEV 2 will be filled by an HDD from VDEV 3. Lastly, the void in VDEV 3 will be filled by a new 4 TB disk.
In the future I can move up to 5 TB HDDs, then 6 TB and so on.
In this manner, I can continually upgrade my storage capacity while still sticking to 24 disks and one server and having great redundancy, even when moving disks from VDEVs.
As I have never used ZFS before I would like to hear if you have any suggestions and see any problems with this approach. Perhaps you know of someone who has done the same thing? I have tried to search for people in a similar situation but without luck - maybe my idea is a bit outside the overton window.
I am thinking about switching to TrueNAS and ZFS permanently. I have installed it twice to test a few things including upgrading HBA firmware.
In the following setup with 8+8+8=24 HDD I would like to hear if what I am planning to do is a smart choice.
VDEV 1:
8 * 1 TB HDD in RAIDZ-3
VDEV 2:
8 * 2 TB HDD in RAIDZ-3
VDEV 3:
8 * 3 TB HDD in RAIDZ-3
In a scenario where disk in VDEV 3 dies I would replace it with a 4 TB HDD. If all 8 disks in VDEV 3 are replaced by 4 TB HDDs I will have 33% additional storage capacity going from 3 to 4 TB per HDD. In other words, a 3 TB HDD gets thrown out and is directly replaced by a 4 TB HDD.
If a disk from VDEV 2 dies, I would remove it from the system and buy a 4 TB HDD. I would then proceed by unplugging a 3 TB HDD from VDEV 3 and then reformat it on another computer. Then I will use that 3 TB HDD and replace the failed disk in VDEV 2. Then I will add the newly bought 4 TB HDD and add it to VDEV 3. In other words a 2 TB HDD gets thrown out (from VDEV 2) and a 3 TB HDD (from VDEV 3) takes over its spot. The 3 TB HDD is taken over by the newly bought 4 TB HDD (new HDD added to VDEV 3).
If a disk from VDEV 1 dies, the same happens as with VDEV 2, but just with one more step. A disk from VDEV 2 will be wiped and made to fill the void in VDEV 1. Then the void from VDEV 2 will be filled by an HDD from VDEV 3. Lastly, the void in VDEV 3 will be filled by a new 4 TB disk.
In the future I can move up to 5 TB HDDs, then 6 TB and so on.
In this manner, I can continually upgrade my storage capacity while still sticking to 24 disks and one server and having great redundancy, even when moving disks from VDEVs.
As I have never used ZFS before I would like to hear if you have any suggestions and see any problems with this approach. Perhaps you know of someone who has done the same thing? I have tried to search for people in a similar situation but without luck - maybe my idea is a bit outside the overton window.