SOLVED Media & Storage (Volume)

[AltecBX]

So I have 12 x 6TB drives in my machine. This server is running FreeNAS 9.3.
I'm looking to use it for Plex media and to backup data as well as retrieve files from it. Do you guys recommend me using 10 drives for Plex (RaidZ2) and using the other 2 drives for data (mirror)?
So in this case I have 10 drives with 43TB for my media & 2 drives with 5.3TB for my data. Or do just recommend me do a big pool with 12 drives in RaidZ2 and then separate them by Dataset?

My chassis holds 16 drives so my plan is to add more drives if my Plex media collection needs to expand as well as my data.

 

[depasseg]

I'd do one pool with 2 datasets.

 

[danb35]

My chassis holds 16 drives so my plan is to add more drives if my Plex media collection needs to expand as well as my data.

You should probably reconsider that plan. ZFS doesn't allow you to add disks to existing RAID sets; you'd need to instead add a new RAID set to it. So, if you went with 12 x 6 TB RAIDZ2, you might add 4 x 6 TB in another RAIDZ2. You'll never be able to change 12 x 6 TB RAIDZ2 into 16 x 6 TB RAIDZ2 without destroying and re-creating the pool.

With that said, I agree with @depasseg--one pool, two (or more) datasets.

 

[depasseg]

Good point, but you also don't want go to 16 drives in a pool.
Performance wise, a single Raid Z2 pool will have the effective write performance of single disk (which isn't great). Another consideration would be a Raid10 Pool. Performance would be much better and it's easier to expand. I assume you are also forecasting the possibility of adding an expansion chassis.

 

[AltecBX]

Thanks guys. I really appreciate your inputs; Depasseg many Thanks to you especially with everything else.
I already installed Plex and was ready to start moving all my movies into this server (14TB). I didn't want to do RaidZ10 because of it eating away at my space. Even RaidZ3 was to much for me. I figure RaidZ2 was a good setup for me maximizing my space and keeping a good performance for what I want to do.

What's should be my best setup in pool for these 12 drives?
Why is 16 drive for a pool no good? I should of bought a 12 drive chassis then.
I might expand in the future especially not that I can pass to another chassis through the backplane J2 port.

 

[depasseg]

Max suggested drives is based on a combination of performance and the amount of time to resilver in the case of a drive failure.
run Z2 and do some performance tests. If it's good enough, then the question of data security needs to be addressed. I'm guessing you won't have backups of all this stuff, will you? If you do, then it's not so bad if you have 2 drive failures and lose the whole pool. If you don't, and you care about your data, you should probably figure out something else. Maybe 3 or 4 RAIDZ1 pools (4 or 3 disks each). That would let you easily add the additional 4 drives in the future and be balance with the rest of the storage.

 

[AltecBX]

I just wiped out both pools. I have all 12 drives now available again. Most of the data is going to be mostly for Plex. (movies, tv shows, and music). I'll have maybe 4-5 TB of data as well. The data will be backup to a drobo device which has 14TB available. The only thing that I won't have a backup of will be the media. And my media grows 20-30GB per week.

This media collection has taken me over 5 years to grow, so no I never want to loose it. I never lost it since I started and I've been using hard drives in my windows machine, then transferred to a Netgear NAS device. And I currently have it on a Drobo NAS device.
I know is possible but do you really think in my environment, that more than 2 drives will die at one time?
Will 12x 6TB RaidZ2 be a preferred setup?

 

[Mlovelace]

For what you're doing with freeNAS I'd go with two 6 x 6tb drive vdevs for your pool. You can always add a jbod later if you need to expand

 

[AltecBX]

A little lost there. What do you mean?
I'm in the Volume Manager page. I want to setup my final pool so I can start testing these drives.

 

[Mlovelace]

A little lost there. What do you mean?
I'm in the Volume Manager page. I want to setup my final pool so I can start testing these drives.

Okay, so drag the icon over 6 and down one. It should show you two raidz2 vdevs with 6 disks in each.

Edit: disks go into vdevs vdevs go into zpools. You can rebuild disks in vdevs but if you loose a vdev you loose the zpool. So, with raidz2 you can loose up to 2 disks in a vdev without loosing the zpool.

I know you want to get up and running but this is the most important time in building a system. You need to carefully plan your zpool because it's very, very hard to change later. Consider future expansion and have spare disks on hand to handle drive failure.

 

[AltecBX]

So with 12 across it gets me 54.56TB at RaidZ2
- With 12 across it gets me 49.10TB at RaidZ3
- With 6 across and 6 below it gets me 43.65TB at RaidZ2
- With 6 across and 6 below it gets me 32.74TB at RaidZ3

Since you guys understand ZFS and FreeNAS more than me; Which one and why?

 

[depasseg]

I think you might want to do some reading on the different RAID levels. Z2 uses 2 drives for parity, Z3 uses 3. So you could lose 2 (or 3) drives and still have your pool in tact. The philosophy goes that the larger the number of devices, the greater the risk of failure, and the risk of failure of a second (or third) failure increases during the rebuild of the first failure. This is because once the first failed drive gets replaced, the pool starts reading all the data from all the other disks to refill (resilver) the replacement disk.

So it boils down to personal willingness to gamble and tolerance for risk.

In the cases where you have 2 rows of 6, what's happening is that each group has 4 data and 2 parity (Z2), or 3 data and 3 parity (Z3) and then those 2 groups are combined.

So in the end:
- 12 across it gets me 54.56TB at RaidZ2 - If more than 2 drives fail, all data is lost
- With 12 across it gets me 49.10TB at RaidZ3 - If more than 3 drives fail, all data is lost
- With 6 across and 6 below it gets me 43.65TB at RaidZ2 - if more than 2 drives in a single group of 6 fails, all data is lost
- With 6 across and 6 below it gets me 32.74TB at RaidZ3- if more than 3 drives in a single group of 6 fails, all data is lost

 

[depasseg]

As for a suggestion, if you have a spare drive available on site and could replace a failure immediately, then the 12 across Z2 isn't too bad. Z3 would be better.

 

[AltecBX]

Ahh gotcha. I understand how raid works.
So after evaluating the 6 across and down raidz2 and the 12 across raidz3, I think I'm going with the 12 across raidz3. I think the 6 across method would just give me a tad bit more protection but at a cost of 6TB. Hard drive availability will be 2 days for me to replace. I can always power down machine in meantime as it doesn't have to be 24/7 if I need to repair. Again the machine is not being used in mission critical enterprise environment but will not like to loose data. In that scenario, 2 days to replace a drive you think I can get away with a z2 setup or you still recommend a z3?


Sent from my iPad using Tapatalk

 

[depasseg]

Z2 is probably ok.

 

[AltecBX]

Mlovelace, I just saw your edit. Will both vdevs be faster than 1 12hd vdev?


Sent from my iPad using Tapatalk

 

[depasseg]

yes. a vdev will write at the rate of the slowest disk (iops wise) in the vdev. If you have multiple vdevs, the load is spread across. So if you went with 2 vdevs, the iops would effectively double.

 

[AltecBX]

Depasseg so in this case the 6 across x 2 makes sense performance wise and hd failure as well. The only thing I'll be losing 6tb as oppose to 12 drives in raidz2.


Sent from my iPad using Tapatalk

 

[depasseg]

6 across x 2 loses 4 drives total compared to 12 across which loses 2 but performance is roughly double.

That's why I'm going with RAID 10 (striped mirrors). I lose 2 more drives that the 6 across x2, but my performance is 6x instead of twice. And I can add drives in pairs if I need to expand.

 

[AltecBX]

Gotcha! I don't think I need super performance since it's basically its going to be stream and transcoding media, but definitely like the safety of 2 drives failing in a 6 hd setup than 12 in 1 pool. What kind of application are doing with your server? And wouldn't I be able to grow as well or will I have to go with a jbod to do so?