Newbie Supermicro build questions

[marksibert]

Sorry for what will probably be dumb questions... I've built plenty of desktop pcs over the years, but this is my first experience with a "real server" and SAS controllers. I've read the HRG, and I'm slowly coming up to speed on FreeNAS and ZFS - but there's a lot to learn.

I purchased a Supermicro X8DTN+ server off ebay (https://www.ebay.com/itm/292576035028). It has a SAS846-7EL1 backplane. It has an Adaptec 5405Z that I will remove. The chassis will support up to 24 drives.

My intention is to initially fill it with twelve 10TB WD RED SATA drives in a ZFS RAIDz2 configuration. Someday, if that fills up, I'll add a second vdev with another 12 drives. The server will have SMB file sharing, a very simple web server, and will run Plex.

So, my questions:

1) Will the SAS846-7EL1 backplane support 10TB SATA drives? I found some conflicting information about that. It sounds like it may, providing the firmware is updated. I think it might be SAS1-based, which the HRG says to avoid. If I can't/shouldn't use it, which backplane should I get? Alternatively, I don't necessarily need the drives to be "hot swappable", so if removing the backplane and attaching directly to the HBA makes more sense, I could do that.

2) What HBA adapter(s) and cables should I get? I see there are only 3 connectors on the backplane. I guess the backplane has a SAS expander integrated into it. Does that mean I only need a single SAS HBA to handle all of the drives? (Assuming I'm using the backplane...)

3) It currently has 72GB of RAM. Is that sufficient for the initial array, which should be approximately 100TB? I do not need compression or deduplication to be enabled. I read the HRG which suggests approximately 1GB RAM for each TB of storage, but I'm not sure what assumptions that is making.

4) For the OS, I see a lot of builds using mirrored USB thumb? (Although not recommended by the HRG.) I have a laptop SSD I could repurpose and connect it to a SATA port on the motherboard? Has anyone done that with a SuperMicro chassis? Is there a way to mount it in the empty space around the motherboard (so it doesn't take up a hotswap bay)?

Thanks for any help/advice!

 

[Chris Moore]

My intention is to initially fill it with twelve 10TB WD RED SATA drives in a ZFS RAIDz2 configuration. Someday, if that fills up, I'll add a second vdev with another 12 drives. The server will have SMB file sharing, a very simple web server, and will run Plex.

You should group the disks by 6 for best performance, 8 if you really must, but 10 is the maximum for RAIDz2 and you loose performance with so many disks in a vdev. It would be better to have two vdevs of 6 drives each in the pool than to have one vdev of 10 drives and each vdev should have the same number of drives.

 

[Chris Moore]

I've read the HRG

What is that?

Oh, I get it, you invented your own acronym... I hate acronyms, of any kind, but new ones most of all...

 

[Chris Moore]

1) Will the SAS846-7EL1 backplane support 10TB SATA drives? I found some conflicting information about that. It sounds like it may, providing the firmware is updated. I think it might be SAS1-based, which the HRG says to avoid. If I can't/shouldn't use it, which backplane should I get? Alternatively, I don't necessarily need the drives to be "hot swappable", so if removing the backplane and attaching directly to the HBA makes more sense, I could do that.

What makes sense is to change that SAS1 backplane for a SAS2 backplane that has an integrated SAS expander so you can have a single SAS controller run all the drives off one cable instead of having to run a separate cable to each drive.
This is the part you want:
https://www.ebay.com/itm/Supermicro...S-SATA-Backplane-24-Port-w-Frame/352372244772

 

[Chris Moore]

2) What HBA adapter(s) and cables should I get? I see there are only 3 connectors on the backplane. I guess the backplane has a SAS expander integrated into it. Does that mean I only need a single SAS HBA to handle all of the drives? (Assuming I'm using the backplane...)

Yes, if you use an expander backplane, you only need one SAS HBA. One like this would do the job:
Dell H310 6Gbps SAS HBA w/ LSI 9211-8i P20 IT Mode for ZFS FreeNAS unRAID
https://www.ebay.com/itm/Dell-H310-...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162834659601
As for the cables, you just need one cable (4 lanes in one cable) like this:
Internal Mini SAS to Mini SAS Cable - NEW/SEALED - 15 INCHES
https://www.ebay.com/itm/Internal-Mini-SAS-to-Mini-SAS-Cable-NEW-SEALED-15-INCHES/352129800027

You will need to look at the length to ensure that it is long enough, and you can use two if you want, but with disk drives you won't need the bandwidth.

 

[Chris Moore]

3) It currently has 72GB of RAM. Is that sufficient for the initial array, which should be approximately 100TB? I do not need compression or deduplication to be enabled. I read the HRG which suggests approximately 1GB RAM for each TB of storage, but I'm not sure what assumptions that is making.

You should be fine with the memory you already got with the system. You will need to see what the demand is once the system is being used in the intended application, but based on what you said, this is probably more than you need.

 

[Chris Moore]

4) For the OS, I see a lot of builds using mirrored USB thumb? (Although not recommended by the HRG.) I have a laptop SSD I could repurpose and connect it to a SATA port on the motherboard? Has anyone done that with a SuperMicro chassis? Is there a way to mount it in the empty space around the motherboard (so it doesn't take up a hotswap bay)?

Sure. That is a great idea. There is a bracket that goes inside the chassis next to the system board. Looks like this:

That allows you to install up to two drives in that space.
You can get that kit (cheapest I can find) here:
https://www.neweggbusiness.com/Product/Product.aspx?Item=9B-16-101-828

 

[marksibert]

Wow!! Thank you so much for the quick replies and great explanation! I think I'll plan on 8 drives in the vdev. Then I can have 3 groups of 8 eventually. That also cuts 1/3 off my initial drive cost. :)

Ordered the backplane and HBA. Need to double-check how long of a cable I'll need. I'll probably get the bracket you mentioned for the SSD too.

 

[Chris Moore]

Wow!! Thank you so much for the quick replies and great explanation! I think I'll plan on 8 drives in the vdev. Then I can have 3 groups of 8 eventually. That also cuts 1/3 off my initial drive cost. :)

Ordered the backplane and HBA. Need to double-check how long of a cable I'll need. I'll probably get the bracket you mentioned for the SSD too.

I forgot to mention, you will probably need one of these things to connect that drive by the system board.

I had one laying around my house last time I needed one, but I thought I would mention it, just in case you need to obtain one.

 

[rvassar]

I forgot to mention, you will probably need one of these things to connect that drive by the system board.

I had one laying around my house last time I needed one, but I thought I would mention it, just in case you need to obtain one.

Some of the new HGST drives will fail to spin up when connected to a Molex to SATA power converter. I do not know if WD is adopting this "standard", but given that they own HGST now, it's possible it could appear at random on WD labelled drives.

For more info see:

https://www.hgst.com/sites/default/files/resources/HGST-Power-Disable-Pin-TB.pdf


-Rob

 

[Johnnie Black]

Some of the new HGST drives will fail to spin up when connected to a Molex to SATA power converter. I do not know if WD is adopting this "standard", but given that they own HGST now, it's possible it could appear at random on WD labelled drives.

For more info see:

https://www.hgst.com/sites/default/files/resources/HGST-Power-Disable-Pin-TB.pdf


-Rob

It's the other way around, they'll fail to spin up when connected to a SATA plug with 3.3v line connected, they'll spin up if using a molex adapter since there's no 3.3v line.

 

[sfcredfox]

If he got the 24 port SAS2 backplane, then the drives will be powered by that, right? There should just be some power connectors allowing the chassis power supply to power the backplane and connected drives I think.

Perhaps you guys were referring to the two side mounted SSDs?

I missing something?

 

[sfcredfox]

this is my first experience with a "real server" and SAS controllers

You might also like this post if you haven't seen it:
https://forums.freenas.org/index.ph...imer-on-basic-sas-and-sata.26145/#post-165190

It was super helpful for me

 

[Chris Moore]

Perhaps you guys were referring to the two side mounted SSDs?

That is what I was talking about when I suggested the need for the adapter. All the power plugs from the power supply on that model server are the old 4 pin Molex. I have a very similar model chassis myself and I don't recall there being even one SATA power connector. I used a Molex to SATA splitter to give me the two power connections I needed for my boot drive mirror.

 

[Chris Moore]

Some of the new HGST drives will fail to spin up when connected to a Molex to SATA power converter. I do not know if WD is adopting this "standard", but given that they own HGST now, it's possible it could appear at random on WD labelled drives.

For more info see:

https://www.hgst.com/sites/default/files/resources/HGST-Power-Disable-Pin-TB.pdf

-Rob

That is actually intended to be a feature but it will cause a few problems to be sure between now and when it is fully supported as a standard. Something to be mindful of, but not (I think) applicable in this particular instance as I was suggesting the power adapter for use on their boot drive which will be an older 2.5 inch SSD that they already have.