Recommendations for a 12-16 Bay Rackmount Server

[scruzloose33]

I started another thread regarding using a large chassis, but I don't want to take that thread too far off topic. I have decided that a 24-bay server is simply too large for what I will likely ever need, so I would prefer to find a 12-16 bay server that will meet my needs. I am open to buying only the chassis and building the rest, but I also wouldn't be opposed to a slightly older server that already has a reliable motherboard and a processor that will meet my needs. I plan to have roughly 30-60 TB of usable storage in RAIDz2, starting with a 6x8TB WD Red vdev, so I assume I will need 32-64 GB of ECC RAM. The primary usage will be for backups of workstations and virtual machines, IP camera NVR storage, and eventually a Plex server (ideally with support for transcoding).

I have read through many of the hardware recommendation threads, but they seem to point primarily to desktop chassis solutions (e.g., Fractal Define R6) or they simply reference using a SuperMicro 3-4U chassis. I am not trying to avoid doing additional research on my own, but I wondered if this community has any recommendations regarding a good chassis (or entire server) that they have seen for a reasonable price recently. There are a large number of SuperMicro servers for sale on eBay, but they range from $200-1500+ and I want to make a good decision for my first FreeNAS build.

 

[Chris Moore]

I have read through many of the hardware recommendation threads, but they seem to point primarily to desktop chassis solutions (e.g., Fractal Define R6) or they simply reference using a SuperMicro 3-4U chassis.

Fractal Define R5 is actually preferable to the R6 because of the way they changed the drive trays in the R6. If you want hot-swap bays though, a used server chassis is the least expensive way to get there. The SuperMicro systems are usually recommended because they are easier to reconfigure as the components are pretty standard where the Dell and HP servers use proprietary system board that you can not change. With a SuperMicro chassis, once you have the chassis, you can change the system board out to upgrade to better hardware and keep the chassis. With a Dell or HP, if you want a newer processor, you can't change the system board, you must buy an entirely new system. That is part of their marketing strategy, they want you to need to buy a new system so you have to keep giving them more money.

I have decided that a 24-bay server is simply too large for what I will likely ever need, so I would prefer to find a 12-16 bay server

The size difference between a 24 bay and a 16 bay is only a couple inches (1.6 if I recall) because a 16 bay server us usually 3U where a 24 bay is usually 4U, rack units that is, and that is not really significant. A 16 bay chassis may be less expensive to get into initially than a 24 bay unit. The advantage of SuperMicro chassis though, you can keep using them for a long time. I have one that is still going after more than 12 years and is on the third system board. It can be an investment that you never really need to replace. Kind of like buying a house.
I will take a look at what is out there for you, but you might find better deals after Jan 1 because that is when companies dump their excess inventory from the previous year.

 

[JohnK]

I have read through many of the hardware recommendation threads, but they seem to point primarily to desktop chassis solutions (e.g., Fractal Define R6) or they simply reference using a SuperMicro 3-4U chassis. I am not trying to avoid doing additional research on my own, but I wondered if this community has any recommendations regarding a good chassis (or entire server) that they have seen for a reasonable price recently. There are a large number of SuperMicro servers for sale on eBay, but they range from $200-1500+ and I want to make a good decision for my first FreeNAS build.

I would recommend Supermicro 3U. I have 3x3u servers in my rack, all bought on Ebay. You can pick up older ones cheaply when searching for complete solutions, for example

Compellent for $100. Rip out the internals and resell on Ebay..

Supermicro 3U CSE-836 Chassis BPN-SAS2-836EL Backplane 6GBPS LSI SAS2X28 US $200 (bought about 2 months ago)

Coraid SR1621 Supermicro X7DBE Xeon 1.86GHz 16 Hard Drive Bay 3U Storage Sever US $70.00 (2017)

 

[Chris Moore]

You can pick up older ones cheaply when searching for complete solutions, for example

Compellent for $100. Rip out the internals and resell on Ebay..

That really is a good deal.
If I didn't have six servers already, and a wife that wants me to sell some of the ones I have, I would buy those.
@scruzloose33 all you would need to do is pull those cards and the system board out and put a new board, CPU and RAM in along with a SAS controller.
I can't tell from the photo but it is likely a SAS2 backplane with an integrated SAS Expander chip.

 

[JohnK]

I can't tell from the photo but it is likely a SAS2 backplane with an integrated SAS Expander chip.

I have one of those. It came with BPN-SAS2-836TQ which means he would have to add breakout cables. That is not a bad option for 8 drives. He can always get the BPN-SAS2-836EL on Ebay. I did that with my first one...

 

[scruzloose33]

I have one of those. It came with BPN-SAS2-836TQ which means he would have to add breakout cables. That is not a bad option for 8 drives. He can always get the BPN-SAS2-836EL on Ebay. I did that with my first one...

I was able to grab both of those Compellent servers off eBay for $150 total. Now I need to figure out what to do with them. It looks like the backplane is the SAS-836TQ, which doesn’t appear to have an integrated SAS expander. Is there any good reason (beyond cleaner wiring) to spend hundreds of dollars to replace the backplane or can I just get a SAS expander card or even connect directly to the motherboard with reverse breakout cables?

 

[Ericloewe]

The TQ backplanes work, but they're a cabling nightmare with individual SATA connections. -A backplanes are much more sane and -EL1 backplanes might make sense if you can get a good deal.

 

[JohnK]

The TQ backplanes work, but they're a cabling nightmare with individual SATA connections. -A backplanes are much more sane and -EL1 backplanes might make sense if you can get a good deal.

You can pick up a EL1 for reasonable prices on Ebay. Been watching one, not that I need it... And it just sold out :(

I actually have one 836 where I kept the 836TQ Back pane. Though it is a bit of a wiring nightmare, I'm running ESXI on it and don't want to pass all the drives to FreeNAS...

 

[scruzloose33]

It looks like these backplanes with built-in expanders run $250-350. I may first try my hand at the complex cabling. I assume I will replace the motherboard so I can upgrade the CPU and get to at least 64G of RAM. Will I also need an HBA/controller and a SAS expander or will some boards enable me to connect directly to the 16 SAS ports on the backplane?

 

[JohnK]

It looks like these backplanes with built-in expanders run $250-350. I may first try my hand at the complex cabling. I assume I will replace the motherboard so I can upgrade the CPU and get to at least 64G of RAM. Will I also need an HBA/controller and a SAS expander or will some boards enable me to connect directly to the 16 SAS ports on the backplane?

If you are patient, you will find a good secondhand one on Ebay for under $100. I would recommend two HBA card. Just bought one of these. (Note you would need different bracket)

Also, when you do buy the breakout cables, shop around. You want longer cables, around 3". And look for cables where the SATA connections are at the right angle.

Also, if you are going for something like the SuperMicro X11-SSM, you will need at least two fan extension cables (CBL-0296L) and possibly and a longer CBL-0071L cable.

Lastly, I replaced my power supplies with platinum, just cause I found some cheap ones on Ebay...

 

[Elliot Dierksen]

I know Supermicro is the most popular platform here, but I have two FreeNAS units based on Cisco C240-M3S chassis. The motherboard supports E5 and E5 v2 CPU's, and has 24 2.5" hot swap SAS/SATA bays. The challenging part can be finding the hot swap sleds, but then can usually be found for ~=$10 each on eBay. It is a 2U chassis, and I have been happy with mine. Disclaimer - I use this kind of hardware at work, so I am very familiar with it.

 

[scruzloose33]

If you are patient, you will find a good secondhand one on Ebay for under $100. I would recommend two HBA card. Just bought one of these. (Note you would need different bracket)

Do you recommend two HBA’s simply for redundancy or is there a performance benefit or necessity?

Would the BPN-SAS836-EL1 work or do I need the SAS2 model? I am having a hard time identifying all differences, but it appears to be a difference between 3 Gbps and 6 Gbps. I found this for a decent price:

https://www.ebay.com/itm/Supermicro-3U-Server-16-Port-SAS-SATA-Backplane-SAS-836EL1/401026075216

 

[JohnK]

Do you recommend two HBA’s simply for redundancy or is there a performance benefit or necessity?

Would the BPN-SAS836-EL1 work or do I need the SAS2 model? I am having a hard time identifying all differences, but it appears to be a difference between 3 Gbps and 6 Gbps. I found this for a decent price:

https://www.ebay.com/itm/Supermicro-3U-Server-16-Port-SAS-SATA-Backplane-SAS-836EL1/401026075216

Most of the cheap HBAs can breakout to 8 SATA drives. So you would need two for 16 drives. You could also do an 16drice HBA, but I would recommend starting with one 8 drive hba and keep looking for a port expander,

I also looked at that backplane. I haven’t done reading on this lately, but I believe it is limited to 2tb drives.

 

[Ericloewe]

You definitely do not want to waste money on anything SAS1.

 

[scruzloose33]

I also looked at that backplane. I haven’t done reading on this lately, but I believe it is limited to 2tb drives.

Can I assume that since the SAS-836TQ backplanes are just passthrough devices that they should not impose any drive size limitations that I may get from a SAS1 backplane? I am going to continue to try to source a reasonably-priced SAS2-836EL1, but I may run out of patience and end up just trying to wire up the TQ backlane from two separate HBAs.

 

[Ericloewe]

As far as the disks are concerned, the -TQ and -A backplanes are a set of wires back to whatever is driving them.

 

[JohnK]

Can I assume that since the SAS-836TQ backplanes are just passthrough devices that they should not impose any drive size limitations that I may get from a SAS1 backplane? I am going to continue to try to source a reasonably-priced SAS2-836EL1, but I may run out of patience and end up just trying to wire up the TQ backlane from two separate HBAs.

That is correct. My recommendation would be to start with only one HBA with 8 breakouts to SATA. If you cannot find a cheap SAS2836EL, you can add another card. Only do that when you want to increase space.

 

[scruzloose33]

That is correct. My recommendation would be to start with only one HBA with 8 breakouts to SATA. If you cannot find a cheap SAS2836EL, you can add another card. Only do that when you want to increase space.

Sorry for the basic question, but is this the style breakout cable that I need to connect from the LSI SAS9211 to the 836TQ? It looks like the connectors on the backplane are 7-pin SATA connectors, but that seems odd to me since it is a SAS backplane. I assume that is because the power is provided separately.

Other than the recommendation to make sure I get long enough cables, are there any other recommendations for good forward breakout cables? Does anyone have experience with the Cable Matters breakout cables like the one below? Am I better off considering right-angle 7-pin connectors for the backplane? This seems like a basic thing, but I would love to get it right the first time so I can start building this weekend.

https://www.amazon.com/Cable-Matters-Internal-SFF-8087-Breakout/dp/B012BPLYJC/

 

[Ericloewe]

Sorry for the basic question, but is this the style breakout cable that I need to connect from the LSI SAS9211 to the 836TQ? It looks like the connectors on the backplane are 7-pin SATA connectors, but that seems odd to me since it is a SAS backplane. I assume that is because the power is provided separately.

Quote:

The TQ backplanes work, but they're a cabling nightmare with individual SATA connections.

Yes, the cables are right. Unless they're mislabeled, and that's always a risk with these.

 

[JohnK]

Sorry for the basic question, but is this the style breakout cable that I need to connect from the LSI SAS9211 to the 836TQ? It looks like the connectors on the backplane are 7-pin SATA connectors, but that seems odd to me since it is a SAS backplane. I assume that is because the power is provided separately.

Other than the recommendation to make sure I get long enough cables, are there any other recommendations for good forward breakout cables? Does anyone have experience with the Cable Matters breakout cables like the one below? Am I better off considering right-angle 7-pin connectors for the backplane? This seems like a basic thing, but I would love to get it right the first time so I can start building this weekend.

https://www.amazon.com/Cable-Matters-Internal-SFF-8087-Breakout/dp/B012BPLYJC/

it should work be fine. The last pair I bought is https://www.amazon.com/gp/product/B000FBT47E/ref=oh_aui_search_detailpage?ie=UTF8&psc=1

I also bought
https://www.amazon.com/gp/product/B011W2F626/ref=oh_aui_search_detailpage?ie=UTF8&psc=1 for cabling my backplane when I replaced it.

One other thing to note is that if you want to wait on replacing the backplane, you could use normal SATA cable from the Motherboard in the interim. For example, if you decide on the Supermicro X11SSM, it has 8 SATA ports and you could cable those to the SAS836TQ backplane.