Best options for 20 x 4TB HDDs

[dahlia]

Hi guys, I'm new to FreeNAS and NAS in general but I've been reading these forums for the past week and I'm pretty confident with my hardware selection so far as I've chosen parts from the recommended hardware sticky.

My question is I'm planning on filling my 20 hot swappable bay case with 4TB drives. I'm aware It's not wise to add all these disks into the same pool so what do you recommend I do?

My case has 4 cages of 5 hard drives. I was originally thinking of making it 4 separate pools as that's cosmetically easier to look at but I've read that raidz2 requires a minimum 6 disks.

Any advice would be great.
Thank you
:)


Sent from my iPhone using Tapatalk

 

[jgreco]

You can add a hundred drives to a pool - or more. You just don't want them all in the same vdev. Cyberjock has a slideshow explaining lots of related stuff over in the New User forum, where this message has been moved to.

 

[dahlia]

Ah sorry for putting the thread in the wrong forum, my mistake. And yeah I just realised I used the wrong terminology.

As you assumed I did mean vdev. How do you recommend I set this up? 4 vdevs of 5 hard drives or something more obscure?

Thanks again.


Sent from my iPhone using Tapatalk

 

[depasseg]

If you are looking for max size, I'd go with 2 RAIDZ2 vdevs (10 disks in each).

 

[dahlia]

Yeah I am looking for max size. I've currently got 24TBs of data on my PC that isn't backed up. I want to set this new NAS box up and have room to transfer all this data onto and then some more.

Is it risky using 10 X 4TB drives in a single vdev?
Maybe I should do 5 x 4TB drives as a raidz3

What do you think about 5 zpools, 4 drives in each at raidz2?

 

[Robert Trevellyan]

Yeah I am looking for max size.
Is it risky using 10 X 4TB drives in a single vdev?

The more drives you have, the sooner you'll see a drive failure, statistically speaking. However, with RAIDZ2 vdevs and a proper SMART testing schedule and properly configured email alerts, you should be able to replace any failing drive long before your pool is threatened.

Going to 4-drive RAIDZ2 vdevs will give you much less storage for the same number of drives, which doesn't meet your goal of "max size".

Do not confuse pools with vdevs. Read cyberjock's slideshow.

 

[dahlia]

Yeah I assumed more drives = more chance of error, thats why I was asking, thank you for the advice.
I understand I'd have less disk space going the 4-drive RAIDZ2 method, I was just trying to think of "safer" ways of going about what I'm doing.

I did read cyberjocks slideshow, I might of confused zpools with vdevs earlier but I really don't want 1 zpool with 20 drives in it, that worries me, I'd rather 2 zpools that are separate from each other.

 

[depasseg]

do you have an easy way to separate your storage needs into 2 halves? Not sure I understand the need for 2 pools.

What it the most important thing to you:
Available usable storage space?
Performance?
survivability from disk failure

Do you plan to back up this box to something else? Or is this your only copy of data? Or is that why you are considering 2 zpools (one for replication)?

 

[dahlia]

The only reason I was thinking about 2 pools was in case an entire array was lost I don't lose the full 20 disks worth of data. Only half of it.

Most important thing for me is both storage space and survivability from disk failure. I've already got 20TB of data on my PC I don't want to lose.

I was originally just going to build the 20 disk NAS box and transfer all my data over to it. But after reading a million posts saying "RAID isn't a backup" I'm a bit hesitant in just copying everything over to it. Maybe my best option would be to keep my 20TBs of data on my PC and use the RAID as my backup.

I was hoping to just have one server in my house with all my data on it and replace my desktop pcs with smaller hard drives. But now I'm second guessing that.

I'm assuming I'm going to have to build two servers to have a proper backup. Offsite backups aren't really an option for me at the moment because my Australian Internet speed is horrible and it would take a year to sync it up with a cloud service.

 

[cyberjock]

The only reason I was thinking about 2 pools was in case an entire array was lost I don't lose the full 20 disks worth of data. Only half of it.

So use RAIDZ3. If things are so broken that the outcome is different by choosing a reasonable RAIDZ3 vdev size versus 2 pools of RAIDZ2, you've failed to setup your server properly and losing your data was an inevitable fact.

 

[dahlia]

So if I chose RAIDZ3, 10x 4TB disks would give me 28TB of usable space... Hmmmm, trying to work out if I really need that 3rd disk and sacrifice survivability for space....

I can just imagine you FreeNAS veterans rubbing your heads reading what I've posted but I honestly really appreciate the answers and advice.

I was also considering setting up 6x 4TB disks as RAIDZ2, then adding another 6x 4TB disks as another RAIDZ2 and then I still have space for one more 6 lot of 4TB drives plus a few spaces in the case. This would be a cheaper alternative by not needing 10 disks to start off with....

 

[depasseg]

So if I had 20 4TB drives and 24 TB of data to backup (and wanted to protect it as much as possible), I would create 1 pool of 10 using RAIDZ2 and use that as the primary. And then I would create another pool with the remaining 10 drives and use it for replication. I would eventually plan to move those ten drives to a different server, when funds became available, to enable offsite (even if off site is a different part of your location). And I would use the scrubs and SMART test scheduling as mentioned above. And I would have a 4TB spare sitting on a shelf in case there was a drive failure.

 

[Robert Trevellyan]

... after reading a million posts saying "RAID isn't a backup" I'm a bit hesitant in just copying everything over to it. Maybe my best option would be to keep my 20TBs of data on my PC and use the RAID as my backup.

Yes, this makes a lot of sense if you don't plan to have a backup of your FreeNAS.

@depasseg 's suggestion of two pools where the 2nd replicates the 1st also makes a lot of sense. Managing two pools is less convenient but adds versatility. I think the question of reliability of one pool vs two is moot if both pools use RAIDZ2 or RAIDZ3. If you had a pool of multiple 3-disk RAIDZ1 vdevs, loss of one vdev taking the pool with it would be much more of a concern.

I was also considering setting up 6x 4TB disks as RAIDZ2, then adding another 6x 4TB disks as another RAIDZ2 and then I still have space for one more 6 lot of 4TB drives plus a few spaces in the case. This would be a cheaper alternative by not needing 10 disks to start off with....

This would be a great way to grow your storage over time. You could add each 6-disk vdev to the original pool (more convenient) or create additional pools (more versatile). You could also replace the original 6 4TB drives one at a time with 6TB or 8TB drives to grow the pool in place. Either way, don't plan to fill your pool more than 80% if you want to maintain decent performance.

Given your lack of experience with FreeNAS and your uncertainty over how to plan your storage, I'd take another step back and start with one smallish vdev and play with it for a couple of months before committing anything important to it or deciding on a specific storage layout. You could even set up a toy system in a virtual machine, but that's more limiting.