FreeNAS Custom-Built NAS for Plex Media Server

[Guinea]

RAID is NOT a substitution for a backup. You have basically four choices. Mirror, 1 disk fault tolerance, 2 disk fault tolerance, and 3 disk fault tolerance. It all depends on how you want to gamble - can you replace a disk in RAIDZ1 quickly and hope that another disk doesn't die while the new disk is being resilvered into the array. Or would you rather go with RAIDZ2 to allow for two disk failures before you lose your data on the third failed drive. It's all in how much space you are willing to give up to keep your data intact. But then again, what if you have a flood or fire? It doesn't matter which RAIDZ configuration you went with, that computer is toast! Only an off-site backup will save you there.

Ok that makes sense. I want it to be as secure as possible....without taking into consideration a floor or fire. This will be sitting on the 2nd floor and if there is a fire....my Plex server will be the last of my worries, haha. So knowing that, I would like to have a decent trade off of hardware failures and available storage. If I did 8 4TB drives in a single vdev (initially), and used Z2, what would be my available storage for use? I think that would determine if I want to go that route.

 

[diedrichg]

If I did 8 4TB drives in a single vdev (initially), and used Z2, what would be my available storage for use? I think that would determine if I want to go that route.

RAIDZ calculator
https://jsfiddle.net/Biduleohm/hfqdpbLm/10/embedded/result/

My spreadsheet to optimize $/TiB.
https://forums.freenas.org/index.php?threads/38092/

 

[Guinea]

Although the more I read (tangent), the more I am leaning towards a Supermicro chassis instead. Apparently NORCO chassis' are a nightmare to rack properly and not very well supported. Unless I went with shelves for all of my components. I am getting the Middle Atlantic RCS-3524 so I am trying to get the best use out of that.

 

[Guinea]

RAIDZ calculator
https://jsfiddle.net/Biduleohm/hfqdpbLm/10/embedded/result/

My spreadsheet to optimize $/TiB.
https://forums.freenas.org/index.php?threads/38092/

Thanks! I think that works fine. I think 18TB should be fine for me initially. I think that is a good start for me. Just trying to settle on the chassis now.

 

[Guinea]

I would look at my build thread ;)

https://forums.freenas.org/index.ph...24-supermicro-x10-sri-f-xeon-e5-1650v4.46262/

And go with an 8-way Raidz2 vdev made with 4TB drives. Then you add another 8 drives and then another 8, and then you replace the first 8 with 26TB drives ;)

One thing that confuses me is the following:

HBA: IBM ServeRaid M1115 (cross-flashed to LSI-9211-8i)
SAS Cables: 2x 0.5m 4 x SATA to SFF-8087 (Reverse Breakout), 4x 0.65m SFF-8087 to SFF-8087

What is the purpose of that? I have only used SATA ports for HDDs before, so these 2 items are a bit confusing to me.

 

[Dave Hamby]

I'm new to this myself. I think the drill is somewhat like this.

1. Determine the total space you want in a single volume
2. Pick the disk size and RAIDZ level and determine the payload size of each VDEV For each RAID level, there is a recommended range of disks per VDEV.
3. Calculate the number of VDEVS you'll need.
4. Pick the hardware storage architecture SATA or SAS. SATA is good for medium systems like mine. If you are proposing more than 6 disks in RAIDZ2, you should consider SAS storage. Your system sounds like it will have 10 or more disks.
5. Figure out now many controllers you need to run all the disks
6. Figure out how many storage cabinets you'll need to hold all the disks
7. Build system
8. Start FreeNAS
9. Walk through configuration. Tell it the RAIDZ level and the disks making up the volume. I believe it will figure out how to organize them into VDEVS and will propose putting the VDEVS into a ZVOL.

That's the way things appeared to work for my medium-sized system. The configurator just knew what to do and asked me to bless it. It may be that easy for you once your build is ready to run.

I think you'll find the following references handy. There are numerous forum threads on used server chassises and and the merits of various SAS controllers and configurations.

1. https://forums.freenas.org/index.php?resources/hardware-recommendations-guide.12/
2. https://doc.freenas.org/9.10/storage.html
3. https://forums.freenas.org/index.php?resources/proper-power-supply-sizing-guidance.39/
   
4. https://forums.freenas.org/index.ph...-sas-sy-a-primer-on-basic-sas-and-sata.26145/

The terminology seems to be in flux from the original ZFS terminology to something more akin to that used by enterprise the storage vendors. I kept getting lost among volumes, pools, virtual devices, and physical devices.

I've only had one go at building a small system. I read all the stuff and was confused like you are. I took my best guess at kit, made my order, boxed it all up, and fired it up. Amazingly, FreeNAS configurator knew what to do with this pile of virgin stuff. It found the volumes. I set the RAIDZ configuration, and the code did the rest. I think you'll be OK if you have supported controllers and the right number of disks for the objective size and RAIDZ level.

 

[Guinea]

Thanks for the info! I think I have most of that stuff solidified already. I am just trying to work through a few gotchas. I know the storage setup and I am pretty confident in that hardware build the other guy linked. I like the idea, although I might go for a newer MoBo just for the heck of it. But the general idea for sure. I am leaning more towards a Supermicro Chassis instead of the Norco so that it racks better and is a little better built. The last pieces I was confused about are the 2 items I listed in my last post. I am confused what those are for and how they work in terms of configuration or their purpose.

 

[mouseskowitz]

One thing that confuses me is the following:

HBA: IBM ServeRaid M1115 (cross-flashed to LSI-9211-8i)
SAS Cables: 2x 0.5m 4 x SATA to SFF-8087 (Reverse Breakout), 4x 0.65m SFF-8087 to SFF-8087

What is the purpose of that? I have only used SATA ports for HDDs before, so these 2 items are a bit confusing to me.

The case will have rows of generally 4 disks. Each row will have a single SAS connector (1 channel per drive, 4 channels per connector) instead of the individual SATA connector per drive you are used to. The HBA is a PCIe card that will have SAS connectors on it. The SAS cables connect the HBA to the back plane. You will need to make sure you have the right number of SAS connectors for your case/drive configuration. Hope that helps.

 

[Stux]

The case will have rows of generally 4 disks. Each row will have a single SAS connector (1 channel per drive, 4 channels per connector) instead of the individual SATA connector per drive you are used to

This depends on the back plane.

My case has 6 passive back planes, one per row of 4 drives. An SFF-8087 cable has 4 lanes/ports, each lane the equivalent of a single SATA cable.

The HBA has two SFF-8087 connectors and supports 8 direct attach drives, or more via expanders.

So I need 24 lanes. So two HBAs for 16 and I have 10 SATA ports on the mobo. So I used a reverse break out cable to connect 8 of those ports to the top two drive bays.

It was not necessary to connect all bays initially, but I did anyway.

Also an expander backplane would only need a single cable and HBA, not six (like mine) and 2-3 HBAs, and not 24 cables and ports like traditional SATA.

 

[Guinea]

Thanks for that bit of information. I am leaning towards doing a Supermicro case. They supposedly include the rails and a redundant PSU included, so the increased price might not be as bad as it looks. I am really just wanting to make sure it works for the rack I chose and I would prefer to not be drilling holes in the chassis at all. So if I went with a Supermicro case, I would need HBAs to handle the SAS connectors based on the configuration of the back planes of the chassis.

 

[Stux]

Thanks for that bit of information. I am leaning towards doing a Supermicro case. They supposedly include the rails and a redundant PSU included, so the increased price might not be as bad as it looks. I am really just wanting to make sure it works for the rack I chose and I would prefer to not be drilling holes in the chassis at all. So if I went with a Supermicro case, I would need HBAs to handle the SAS connectors based on the configuration of the back planes of the chassis.

Yes. With an Expander backplane, you would need just one HBA. And if you're buying a SuperMicro motherboard, you can buy one with an LSI HBA included on the board.

 

[Guinea]

Yes. With an Expander backplane, you would need just one HBA. And if you're buying a SuperMicro motherboard, you can buy one with an LSI HBA included on the board.

Oh ok perfect! The one I was looking into was this one: CSE-846A-R1200B. I am trying to look through the specs to make sure it matches my current HDD list and motherboard etc. And see if it has that LSI HBA included.

Doesn't really tell me on their website:

"24-port 4U SAS 6Gbps direct-attached backplane, support up to 24x 3.5-inch SAS2/SATA3 HDD/SSD"

Or am I reading it wrong?

 

[danb35]

The one I was looking into was this one: CSE-846A-R1200B

That does not have an expander backplane, as the wording you quoted shows--"direct-attached backplane" is not what you'd need. You'd be looking for an 846E16.

 

[Guinea]

That does not have an expander backplane, as the wording you quoted shows--"direct-attached backplane" is not what you'd need. You'd be looking for an 846E16.

So right in my face then haha. Ok, I will keep hunting around. There is probably a similar chassis that has one.

I think this one would meet my specifications: SC846BE16-R1K28B

I just need to go through my hardware list and make sure it is all compatible and matches. If I go with that chassis, what would I need as opposed to your parts list? How would I determine what cables and HBAs I need based on what is included with that chassis and my initial setup of 8 4TB HDDs?

 

[Guinea]

Yes. With an Expander backplane, you would need just one HBA. And if you're buying a SuperMicro motherboard, you can buy one with an LSI HBA included on the board.

So would this meet the requirements of your build but have the LSI HBA?

MBD-X10SRH-CLN4F-O

This + the following chassis: SC846BE16-R1K28B should be a really solid combination, right? Am I missing anything else? Would I need any additional cabling other than what is included?

 

[Stux]

Sweet board :)

Lots of x8 slots. A x4 for an SSD AIC and quad gigabit i350 ethernet :)

Chassis looks perfect too. But I'm not a SM chassis expert

 

[Guinea]

Sweet board :)

Lots of x8 slots. A x4 for an SSD AIC and quad gigabit i350 ethernet :)

Chassis looks perfect too. But I'm not a SM chassis expert

So would I still need an HBA board? I can wait until everything gets here before determining the necessary cables that are missing. But I think that should be it right? Just making sure before I pull the trigger. The chassis also doesn't have USB on the front, but it should be fine to use the boot drives on the rear of the chassis? I am assuming you have 2 to mirror them?

 

[droeders]

So would I still need an HBA board?

No - there is SAS on the motherboard that you can use instead of an HBA.

I can wait until everything gets here before determining the necessary cables that are missing. But I think that should be it right? Just making sure before I pull the trigger.

Here's the URL to the backplane manual for that chassis:
https://www.supermicro.com/manuals/other/BPN-SAS3-846EL.pdf

Looks like it has Mini-SAS (SFF-8643) connectors on the backplane. I'm not certain, but the block diagram on page 1-9 of the X10SRH-CF manual seems to indicate there are SFF-8643 connectors there too.

The chassis also doesn't have USB on the front, but it should be fine to use the boot drives on the rear of the chassis? I am assuming you have 2 to mirror them?

Personally, I would use a single SSD instead of USB. If you do go for USB, I'd definitely mirror them. However, it's not required.

 

[Guinea]

Personally, I would use a single SSD instead of USB. If you do go for USB, I'd definitely mirror them. However, it's not required.

Is there a reason you would suggest going against a USB Boot drive? Would it be fair to assume I would just grab a standard Samsung (i.e.) SSD? Would I plug it into my gaming computer top load the FreeNAS build and then plug it into the NAS after? I was hoping to leverage the IPMI of the motherboard, but I don't know how that all works yet. Still toying with these concepts as I push through the documentation. Thanks again for the help!

 

[Jailer]

Is there a reason you would suggest going against a USB Boot drive?

Because most are crap quality and prone to failure.

I was hoping to leverage the IPMI of the motherboard,

Using this you won't need any install media you just mount the image file and install to the destination drive. Trust me, take the time to learn your IPMI and you'll wonder how you ever got by without it.