BUILD First Build - Advice

[nlaveck]

Hi guys,

I'm new to the forums and FreeNAS. I've been reading about FreeNAS for a while, and finally decided to make the leap because my 3 TB media server is almost full. Here's the hardware I have in mind.

I got excited an already ordered everything... but I'm kind of regretting that now because I'm second guessing a lot of my decisions. I wouldn't be opposed to returning anything if need be and ordering something else. Here's my thoughts on my current build:

• I was originally thinking of doing RAIDZ2, but upon further reading I'm planning on going with Striped Mirrored Vdev’s for added flexibility
• Since I'm going with Striped Mirrored Vdevs, I'm not sure if my best best is starting with 6 3 TB drives since I'd need to add a HBA to expand. (Maybe start with a single mirror of 6TBs?)
• I don't plan on going above 10 drives, but if I do expand to 10 would I need a bigger power supply?

Only thing that's not in the build list is I'm planning on mirroring 2 usb flash drives for my boot device.

Thanks!

 

[Ericloewe]

I was originally thinking of doing RAIDZ2, but upon further reading I'm planning on going with Striped Mirrored Vdev’s for added flexibility

Why are you concerned about this? RAIDZ2 is significantly more reliable and has less overhead.

I don't plan on going above 10 drives, but if I do expand to 10 would I need a bigger power supply?

It would be advisable, yes.

 

[MatthewSteinhoff]

This is for home use, right? A handful of users at most? With few users, throughout isn't really an issue no matter the drive configuration, RAIDZ2 or mirrored.

These days, 3TB drives feel so small for NAS storage. A pair of mirrored 6TB drives will save bays, ports, electricity, complexity and double your existing space. If you want to expand, mirror another two, 6TB drives and toss them into the existing volume group. Keep doing that until you run out of ports/bays.

If your growth rate is low, I'd go the mirrored route. If you plan on immediately needing a lot of space, the large initial RAIDZ2 route would be better because it is more efficient.

The power supply - with six drives - looks small but I haven't done the math. With just two, 450 watts should be fine. I'd still go up a size or two for expansion.

Cheers,
Matt

 

[nlaveck]

Thanks for the quick reply!

Why are you concerned about this? RAIDZ2 is significantly more reliable and has less overhead.

To be completely honest, I'm still pretty new to RAIDZ and vdevs, but from what I've read it seems like once I set up a RAIDZ configuration it's stuck that way unless I move off the data and rebuild the array. So if I wanted to expand beyond that I'd need to add an entirely different configuration to the pool. I think mirrors would be easier to manage, but since you brought up the reliability point I'm starting to sway back to RAIDZ2.

I also read this post and have a few friends using RAID 10 (not ZFS) for things.

 

[nlaveck]

These days, 3TB drives feel so small for NAS storage. A pair of mirrored 6TB drives will save bays, ports, electricity, complexity and double your existing space. If you want to expand, mirror another two, 6TB drives and toss them into the existing volume group. Keep doing that until you run out of ports/bays.

This is what I've been thinking recently after I was swayed away from RAIDZ2. I'm not sure anymore though. After what Ericloewe said I'm on the fence. I'm sort of thinking 3 TB drives was a bad idea, but I jumped at the cheapest RAIDZ2 array with a decent size. 12 TB should last me a good while, but the 6 TB mirrors would have a max size of 18 TB without HBA's. Not sure if the extra 6 GB is worth it or not considering I'd have to return the drives I already bought and overall it'd cost more.

 

[Spearfoot]

Hi guys,

I'm new to the forums and FreeNAS. I've been reading about FreeNAS for a while, and finally decided to make the leap because my 3 TB media server is almost full. Here's the hardware I have in mind.

I got excited an already ordered everything... but I'm kind of regretting that now because I'm second guessing a lot of my decisions. I wouldn't be opposed to returning anything if need be and ordering something else. Here's my thoughts on my current build:

• I was originally thinking of doing RAIDZ2, but upon further reading I'm planning on going with Striped Mirrored Vdev’s for added flexibility
• Since I'm going with Striped Mirrored Vdevs, I'm not sure if my best best is starting with 6 3 TB drives since I'd need to add a HBA to expand. (Maybe start with a single mirror of 6TBs?)
• I don't plan on going above 10 drives, but if I do expand to 10 would I need a bigger power supply?

Only thing that's not in the build list is I'm planning on mirroring 2 usb flash drives for my boot device.

Thanks!

Welcome to the forum!

Hint: you'll get more responses if you post your actual parts list here; many users are too lazy or cantankerous to follow a link... :)

Does the Core i3 CPU you selected support ECC memory? And is the Crucial RAM kit an ECC memory kit?

The PSU may be okay, check the "Proper Power Supply Sizing Guidance" thread to make sure.

Good luck!

 

[Ericloewe]

Does the Core i3 CPU you selected support ECC memory? And is the Crucial RAM kit an ECC memory kit?

Yes on both counts.

 

[joeschmuck]

You should change your mindset just a little bit on how to choose your storage and it's setup.

This is a home setup, right? If yes then here is some guidance I give to people who don't know what to do.

1) Over the next 3 years, how much storage do you plan to need?
2) Double the answer given in question 1.
3) Will you be using a SSD as a boot device?
4) How many SATA ports do you have? If you answered yes to question 3, subtract one SATA port.
5) RAIDZ2 is typically the best compromise of data safety for a home user so we should try to use RAIDZ2 if at all possible. And you should be looking at having either 5 or 6 hard drives if possible, up to 8 if needed to meet the capacity desired.

Rationale: I use 3 years because the typical warranty lasts 3 years. If your warranty lasts longer then you can adjust as appropriate. The reason we double it is because most users underestimate and because ZFS works better with 20% or more free space. Yes, this means you should never try to fill up your storage. Now if you will be using a SSD as a boot device (you only need one and we all highly recommend one) you need to free up one SATA port to support it. If you have an HBA card then of course you don't have to worry about that aspect.

So lets say you desire 10TB of storage, well double that and you have 20TB which is quite a lot. I myself have only 7TB of storage and still have 4.7TB free. If you only have five SATA ports then you will need 8TB drives. This is not ideal due to the resilvering time should you need to replace a drive for failure. Resilvering is the drive recreating the data on the new drive and for very large drives it can take several days to complete. I would go no larger than 5TB drives in a RAIDZ2 format but that is just a personal opinion. In this situation where I desire 20TB of storage, I'd add an HBA so I can attach up to 8 hard drives and then I could use eight 4TB drives to get to 20TB. If you only needed 4 more SATA ports, you could add a SATA port card but you just can't pick anything and hope it works reliably. I can tell you what actually does work and will remain supported by FreeBSD 11 and likely beyond that too, if you need that data. If everything fits on your motherboard ports then you are doing fine.

Once you have your storage setup, if you find yourself running out of space (hopefully after 3 years) then you can replace each hard drive with a larger capacity. Once you have replaced all the hard drives the capacity will jump up to the new size. The capacity will not increase as you insert a larger drive because the data is striped across all the drives and therefore requires all the drives.

Also, knowing how many drives you will need can be a factor in selecting the correct motherboard. It can be cheaper to select a motherboard with enough SATA ports over adding an HBA card.

 

[nlaveck]

Thanks you everyone for your input! think I'm going to try to go with a RAIDZ2 like I initially planned since that'd provide the most storage with decent redundancy at low cost. I might upsize the power supply, but I need to gather some more stats first.

 

[nlaveck]

Once you have your storage setup, if you find yourself running out of space (hopefully after 3 years) then you can replace each hard drive with a larger capacity. Once you have replaced all the hard drives the capacity will jump up to the new size. The capacity will not increase as you insert a larger drive because the data is striped across all the drives and therefore requires all the drives.

Awesome! I didn't know RAIDZ could do that! I thought I'd have to transfer the data to a backup server to upgrade the drives, this makes me feel a lot better about my hardware choices. Is there a way to increase the number of the drives in a RAIDZ without transferring data as well?

 

[joeschmuck]

Is there a way to increase the number of the drives in a RAIDZ without transferring data as well?

Yes and No. If you feel you will need a very large amount of storage and need to run 10+ hard drives, you can basically create multiple vdevs (groups of drives) and then add another full group of drives.

Scenario for Yes: You need 10TB of storage right now and you build it using five 4TB drives. 2 Years later you want to double the storage so you would add five more 4TB (or larger) drives as a vdev and then add the vdev to your other vdev. This create two vdevs joined thus doubling the capacity. You don't have to add the same number of drives, you could add a single drive however if that single drive were to fail, ALL your data would be gone which is why when you add hrives like this, you add another RAIDZ2 vdev so you still have a two drive failure safety net of each vdev. Look at Cyberjocks vdev guide.

The NO answer: So most people would rather not add two more drives for that safety net and would rather rebuild the pool with more drives and likely larger drives too. To accomplish this you must backup all your data first. Then you destroy your pool and then you add your new drives and create the new pool.

Some added advice: You say you have grown out of a 3TB media server. If you truly feel that you would be fine for the next three years running a RAIDZ2 of six 4TB hard drives then you would have ~14TB of storage. If you desired you could run five 4TB drives and have ~10TB of storage.

Once you have figured out how much storage you desire, then choose the motherboard and case. You may really want a compact footprint but with that come airflow and heat concerns, yes, even for the name brand cases. And always plan for the future expansion if you can. Hard drives will fill up. Myself, if I have a movie on my server, it's only because it's something I know I will watch a few times. Anything that was a dud or I won't likely watch it a second time, I delete it.

 

[Robert Trevellyan]

I also read this post

There's a lot of misinformation in that article, which doesn't mean using mirrors is a bad idea, it just means you should ignore just about everything else in it. The argument in favor of mirrors was presented coherently five years earlier: http://constantin.glez.de/blog/2010/01/home-server-raid-greed-and-why-mirroring-still-best