Build advice, Combo or all in one.

[pjuser115]

I built a test pc and I am glad I did. FreeNAS took me a long time (about 5 weeks) to get it up and running. Now that I am ready to build an actual server I have a few more specific questions.

My server will have 10 HDD (3x6TB, 3x3TB, 4x4TB) After looking around at prices I see it is just about the same amount of money to get a server board that has 10+ SATA ports as it does to buy a board with 2-4 SATA and then buy a PCIe addon card with the 8 port SATA.

What are the advantages / disadvantages or either path? I can see an advantage of the SATA ports failing on the PCIe card is easy to replace the card rather than the whole Motherboard. I also assume that the all-in-one option is easier on drivers. I assume PCIe board is no slower than on board SATA so that's probably not an issue. What am I not thinking about, what would be a reason to go 1 way or the other, or is is all the same.

Examples of things I am considering.

OPTION 1:
all-in-one with 10 x SATA 6.0Gb/s $260
SUPERMICRO MBD-X10SRL-F
https://www.newegg.com/Product/Product.aspx?Item=N82E16813182927


OPTION 2:
MoBo and PCIe with 6 x SATA 6.0Gb/s onboard and 4 x SATA 6.0Gb/s from PCIe $265
(taken from Chris Moore https://www.ixsystems.com/community/threads/freenas-home-build.72885/#post-504949)

System Board: Supermicro X9SRL-F Motherboard Socket LGA2011 System Board w/ I/O Shield - - US $206.99
https://www.ebay.com/itm/401678513337

Drive Controller: SAS PCI-E 3.0 HBA LSI 9207-8i P20 IT Mode for ZFS FreeNAS - - US $44.88
https://www.ebay.com/itm/192639052923

Data Drive Cables: Mini SAS to 4-SATA SFF-8087 Multi-Lane Forward Breakout Internal Cable - - US $12.99
https://www.ebay.com/itm/371681252206




I am not set hard on either setup, just looking at both. Really just want to know if I should focus more on all-in-one set up or if it doesn't really matter at all.

 

[2nd-in-charge]

10 HDD (3x6TB, 3x3TB, 4x4TB)

Have you already bought the drives? And have you thought how are you going to setup Zpools with this set?

I can see an advantage of the SATA ports failing on the PCIe card is easy to replace the card rather than the whole Motherboard.

If only the SATA controller fails you can just add an HBA at that point and keep using the motherboard :)

Motherboard ports are also less mucking around (e.g. you don't need to flash the HBA firmware), and marginally less power consumption.

OTOH, X9SLR-F motherboard uses DDR3 RAM, which would be cheaper than DDR4, especially used.

Data Drive Cables: Mini SAS to 4-SATA SFF-8087 Multi-Lane Forward Breakout Internal Cable - - US $12.99
https://www.ebay.com/itm/371681252206

I think you might have to buy sata cables to connect all 10 drives to X10SRL-F .

Also, what about boot drives? Two boot SSDs consume two SATA ports. X9 wins here as well.

 

[Constantin]

You might be better off considering something integrated like the X10SDV-2C-7TP4F. Onboard HBA (16 channels), SFP+ cages for future expansion, NVME m.2 slot at PCIE 3.0x4, 2 PCIe slots, a fast CPU, 4 DIMM slots.

I do not understand your drive choices at all. Consider figuring out one size that works for your pool and then build a VDEV combination / pool around that. For example, a Z2 Pool with six 10TB drives would give you about the same storage but with Z2 redundancy. Or do a Z2 with ten 6TB drives. Your collection of leftover 3,4TB drives is probably best used for backing up critical data offsite - in something like an Oyen Digital Mobius 5 array.

 

[pjuser115]

Have you already bought the drives? And have you thought how are you going to setup Zpools with this set?

I have these drives already. I am not as much worried about redundancy beyond Z1 for the 6TB array, this will be movies, tv, music.
The 4TB array will be Z2 which will be what I care about all family photos, personal files, etc.
the 3TB array will be something not as important like my FTP and whatever share I don't care about.

The stuff we care most about (4TB array) will most likely stay on my current media computer as a 2nd backup since that computer already has a few 3TB drives in it as well. They are just not mirrored so if they fail all is lost, which was the whole point of this freeNAS research for the past 2 months.

I think you might have to buy sata cables to connect all 10 drives to X10SRL-F .

You are probably right I should have factored this in, however I do have about 20 Sata III cables laying around

Also, what about boot drives? Two boot SSDs consume two SATA ports. X9 wins here as well.

It is my understanding that freeNAS only uses the boot device once, on the initial boot? Thus the cost and speed of a SSD would be wasted? I figured I would do 2 or 3 USB drives and save the config files on my gaming PC :) Is this not the right way of thinking?

You might be better off considering something integrated like the X10SDV-2C-7TP4F. Onboard HBA (16 channels), SFP+ cages for future expansion, NVME m.2 slot at PCIE 3.0x4, 2 PCIe slots, a fast CPU, 4 DIMM slots.

Considerably more expensive but I really do want SFP+ for later down the road. Figured I would save the 200 on the motherboard and get a PCIe card when that time comes.

I do not understand your drive choices at all. Consider figuring out one size that works for your pool and then build a VDEV combination / pool around that. For example, a Z2 Pool with six 10TB drives would give you about the same storage but with Z2 redundancy. Or do a Z2 with ten 6TB drives. Your collection of leftover 3,4TB drives is probably best used for backing up critical data offsite - in something like an Oyen Digital Mobius 5 array.

This is all good and proper like you have said, but that would cost me A LOT more. I have all these HDDs already and really only care about 3TB worth of data (currently) I would be sad if I lost my media files but not the end of the world. Z1 should be good enough in this case.

6x10TB HDD Seagate red is 306 and pro is 370. That's an additional $1,836 or $2,220
buying an additional 7x6TB HDDs would be $1,295 or $1,540 respectively

That is a lot of money and wife aggro!

 

[Constantin]

Not a worry. I was under the impression that you needed all of these drives to get to 30TB of capacity. I still would advocate for fewer, large drives in a Z2. The electricity costs of running a drive 24/7 can be substantial.

In other words, unless you actually need to use the drives, I’d leave them disconnected.

 

[pjuser115]

Not a worry. I was under the impression that you needed all of these drives to get to 30TB of capacity. I still would advocate for fewer, large drives in a Z2. The electricity costs of running a drive 24/7 can be substantial.

In other words, unless you actually need to use the drives, I’d leave them disconnected.

Sounds good, I wasn't really planning on hooking up the 3TB array for a while since i will have more than i need with the 6TB and the 4TB arrays.

But back to the original question... it seems like it really doesn't matter one way or the other.... which ever gets the job done and is within however much I want to spend is the option I should go with? PCIe or all in one mobo is all the same in the end?

EDIT:
Can you also touch on this so I can make sure I understand it correctly?

It is my understanding that freeNAS only uses the boot device once, on the initial boot? Thus the cost and speed of a SSD would be wasted? I figured I would do 2 or 3 USB drives and save the config files on my gaming PC :) Is this not the right way of thinking?

 

[2nd-in-charge]

The 4TB array will be Z2 which will be what I care about all family photos, personal files, etc.

3-drive Z2? might as well have a 3-way mirror.
If you want you can setup 6-drive RaidZ2 with your 3 and 4Tb drives. The extra 1Tb on the 4Tb drives will be unused, but you'll get 12Tb storage with double redundancy. Or, if you don't mind moving data around, put the 4Tb drives into the media PC and get the 3Tb drives into the FreeNAS.

It is my understanding that freeNAS only uses the boot device once, on the initial boot? Thus the cost and speed of a SSD would be wasted? I figured I would do 2 or 3 USB drives and save the config files on my gaming PC :) Is this not the right way of thinking?

That's where all the updates go as well. Eventually most USBs will fail, SSDs will last longer. A 120Gb SSDs like Kingston A400 will do the job, and is cheap as chips these days. You're right about saving config files.

 

[pjuser115]

3-drive Z2? might as well have a 3-way mirror.
If you want you can setup 6-drive RaidZ2 with your 3 and 4Tb drives. The extra 1Tb on the 4Tb drives will be unused, but you'll get 12Tb storage with double redundancy. Or, if you don't mind moving data around, put the 4Tb drives into the media PC and get the 3Tb drives into the FreeNAS.

No the 4TB will be 4 drive. I could shuffle drives around and come up with 5x3TB for a Z2 (maybe buy an additional to make it 6x3TB) but i think 4x4TB would be good enough for my "safe" backup.

That's where all the updates go as well. Eventually most USBs will fail, SSDs will last longer. A 120Gb SSDs like Kingston A400 will do the job, and is cheap as chips these days. You're right about saving config files.

I could and I am not against it. But I think it is probably safer and cheaper to use 2 or 3 USBs than the 1 ssd like you mentioned.

 

[2nd-in-charge]

the 4TB will be 4 drive.

You did say that in the opening post. My bad.

I could shuffle drives around and come up with 5x3TB for a Z2

Will give you an extra Tb on the Freenas (3x3 vs 4x2) and extra 2 on the media PC :)