Different sized drives in pool (multiple vdevs)

[g4m3r7ag]

Hello, I am new to FreeNAS but have recently acquired a 45-bay SuperMicro JBOD and have a spare r710 to run FreeNAS. I also purchased seven of the 8TB easy stores that BestBuy had on sale this weekend. I plan to set them up in RaidZ3. My current storage solution is an 8-bay Synology that is fully populated with 4TB WD Reds. They have been in service for almost two years now but according to the Synology they are passing extended SMART tests.

My question is, is it ok to use seven of those drives in another RaidZ3 vdev added to the same pool with the 8TB drives? Or do all the vdevs in that pool need to be 8TB drives? Is it safe to use those drives seeing as they have been in service for nearly two years already? If the drives don't have to be the same size in the pool is it ok for them to be wildly different? For example could I add a seven drive RaidZ3 of 2TB drives to the same pool as the 8TB drives?

This pool will be mainly for Plex media storage. Also I understand the drives in a vdev should be identically sized or the vdev will only allocate space as if all drives were the same size as the smallest in the vdev. Just not sure if that is also recommended across vdevs in a pool.

 

[Jailer]

You can combine differing size vdev's together in a single pool. You only lose the available space on a drive when you combine different sized drives in the same vdev.

 

[SweetAndLow]

Hello, I am new to FreeNAS but have recently acquired a 45-bay SuperMicro JBOD and have a spare r710 to run FreeNAS. I also purchased seven of the 8TB easy stores that BestBuy had on sale this weekend. I plan to set them up in RaidZ3. My current storage solution is an 8-bay Synology that is fully populated with 4TB WD Reds. They have been in service for almost two years now but according to the Synology they are passing extended SMART tests.

My question is, is it ok to use seven of those drives in another RaidZ3 vdev added to the same pool with the 8TB drives? Or do all the vdevs in that pool need to be 8TB drives? Is it safe to use those drives seeing as they have been in service for nearly two years already? If the drives don't have to be the same size in the pool is it ok for them to be wildly different? For example could I add a seven drive RaidZ3 of 2TB drives to the same pool as the 8TB drives?

This pool will be mainly for Plex media storage. Also I understand the drives in a vdev should be identically sized or the vdev will only allocate space as if all drives were the same size as the smallest in the vdev. Just not sure if that is also recommended across vdevs in a pool.

Having different size drives in a pool is perfectly normal. I currently have 2 vdevs, one with 3TB drives and one with 4TB drives and I'll be adding another with 8TB drives soon.

 

[garm]

Unless you stripe the drives (and you shouldn’t) a vdev can only be the size of the smallest drive multiplied with the number of drives and subtracted the parity, and some overhead.
A pool can consist of more then one vdev, but it has no redundancy in that layer. You loose one vdev, you lost your entire pool.

The topography is what’s important. You cannot (yet) change the initial configuration of devices in a vdev or remove a vdev from a pool. Therefor you need to make sure the configuration you choose is something you can live with and fits your expansion strategy.

I myself use mirror pairs in small home applications as they are easy to expand in small increments. Large raidz vdevs are usually harder to buy in a consumer type scenario and I don’t like to sit with large unused disk space as drives age regardless of the utilization. For TCO I try to stay above 50% but well below 80% pool utilization at any time.

When it comes to mixing size and performance disks in a vdev I pay less attention to that. Maybe someone else will bash me for it but I tend to stick to 5600 rpm NAS drives and they usually have equivalent performance. It’s nothing that have bitten me yet any way. Maybe if I one day build something bigger then single digit TB storage servers that will be a measurable factor, but not at the moment

 

[g4m3r7ag]

Thank you for the help with the different sized vdev's. Does anyone have an opinion on using the 4TB drives that have been in service for two years already? Is that too much of a risk? I also purchased them all at the same time from Amazon so the Serials are sequential.

 

[SweetAndLow]

Thank you for the help with the different sized vdev's. Does anyone have an opinion on using the 4TB drives that have been in service for two years already? Is that too much of a risk? I also purchased them all at the same time from Amazon so the Serials are sequential.

If smart tests and info are ok everything should be fine. Drives that have been used for 2 years are probably better than drives right out of the box.

 

[g4m3r7ag]

Thank you, after I move my data to the 8TB drives I'll probably run bad blocks on the 4TB drives and verify everything comes back ok.

 

[nojohnny101]

Thank you, after I move my data to the 8TB drives I'll probably run bad blocks on the 4TB drives and verify everything comes back ok.

That doesn't make sense to me. At the most you could run short and long SMART tests but it seems those were already being done on a schedule. Just keep doing regularly scheduled SMART tests and you'll be fine.

 

[g4m3r7ag]

Yes, the Synology has scheduled quick SMART tests every Sunday and it runs a long test on one disk per day the 1st through the 8th of every month. Also has a scheduled data scrub that runs on the 29th of every month prior to the long test. I guess the biggest part of my worry was them all being sequential serial numbers and dropping 4+ disks before a resliver could be completed if they start failing in another year. I understand raid is not a backup but it is hard to have a backup solution for 20+ TB of media that doesn't cost a small fortune.

 

[nojohnny101]

Yes, the Synology has scheduled quick SMART tests every Sunday and it runs a long test on one disk per day the 1st through the 8th of every month.

Sounds good.

Also has a scheduled data scrub that runs on the 29th of every month prior to the long test.

Ok

I guess the biggest part of my worry was them all being sequential serial numbers and dropping 4+ disks before a resliver could be completed if they start failing in another year. I understand raid is not a backup but it is hard to have a backup solution for 20+ TB of media that doesn't cost a small fortune.

I'm not going to say that is impossible, but darn near impossible. You are running the highest form of redundancy available on FreeNAS, and I'm not statistician but the likelihood of 4 drives failing, I'd bet my life savings that won't happen to you (I like my odds). If you have a solid backup solution in place that meets your needs, then worse comes to worse, you restore from backup (you check the integrity of your backups correct?).

Running bad blocks is not going to mitigate or given you any peace of mind logically with what you're afraid of (4 drives dying at the same time). If you really wanted to, you could cut down on time and keep a cold spare, that might give you peace of mind. This way you wouldn't have to wait around for a new drive to arrive or have the RMA be the bottleneck to brining your pool out of a degraded state.

 

[g4m3r7ag]

I actually have no backup solution for the largest chunk of my storage. The important things, photos, important documents etc, are all in multiple locations on multiple systems and in cloud storage as well. However the largest part of my storage, Video media, has no backup other than the raid. Do you have a recommendation for backing up 20TB without breaking the bank? I just spent what funds I had available to build the FreeNAS machine and populate it with enough 8TB drives to increase the space available from what I have on my Synology currently with eight 4TB drives. I am down to just over a TB of free space on it with two disk redundancy.

 

[nojohnny101]

Well you're not the only one. Although most on here, including myself, advocate for a backup (as you mentioned, RAID or redundancy is not a replacement for a backup) solution, I'm also very practical and realize that this might not be in the budget for everyone.

It sounds like you have already backed-up what's important, so that's good. I mean you really don't have a choice for the other media if you don't have the budget. The cheapest route is cheap USB external drives but that would not really be practical with 20TB of data. You're fighting the economics of scale. You could build a barebones backup FN box and while the motherboard, ram, CPU, case, power supply won't cost that much, buying the drives necessary to hold 20TB of data for backuping up (with planning for future storage needs in mind) will likely be cost prohibitive.

I am down to just over a TB of free space on it with two disk redundancy.

This is on your main box? How full (percentage) is your main pool?

A last thought, you might want to reevaluate what you are storing. I'm not going to pretend to know what your storing, but I have talked to people previously that were running into storage issues and didn't have the money to keep expanding, so they just started deleting media they rarely asked and wasn't important (i.e. movies you haven't watched for years or only watched once and don't hold sentimental value). Every person I talked to didn't regret pruning after they had done it.

 

[g4m3r7ag]

It's ~90% full but it is not ZFS it is a Synology device. It recently died and even though they replaced it the RMA was process was enough to convince me to not just replace the 4TB drives in it with eights and build a FreeNAS box for the eights. Rather than sell the Synology after I move the data I may just start buying a 10-12TB drive every couple of months and by this time next year have a decent sized pool on the Synology in SHR-2 for archival backup. I had the money to expand just not expand and create a separate backup all at once. I could probably prune a decent chunk of data but the data hoarder in me can't do it lol