TrueNAS 24/36bay 4U Server 2022 Edition

[Fireball81]

Hey there,

i am looking for a new 24/36bay, 4U DIY TrueNAS Build.
The idea is to start with 12x18TB Seagate Exos in a single RAID-Z2 VDEV and expand from there, if need be.
It should be only used for backups but in case our main server fails, it should be able to take over, at least for a short period of time, so
10GbE is what i am looking for here and it should be reasonably fast.

24 bay will suffice but 36 bays for some extra headroom is nice to have so it depends on the case i can get for a decent price.
If the case FANs are bigger (120mm+) and not super loud, thats a plus, because we have an office room close to the server cabinet, so i gotta keep that
in mind.

I have some ideas in terms of hardware choice but i would like to double check with what you guys would recommend, right now.
Especially the case option is a little bit tricky at the moment. (Availability, price and so on)

For example:
I can get the SuperMicro CSE-846TQ with a BPN-SAS-846TQ passthrough backplane for a reasonable price.
Then there is LogicCase. Does anyone here have experiences with their 4U enclosures, like for example

4U Server Chassis

Logic Case 4U rackmount chassis are perfect for storage-dense applications, or where a lot of storage drives are to be installed, either in a fixed or hot-swap format. Our 4U chassis come in a diverse range of depths and feature-sets, including super short depth, all the way through to full...

www.logic-case.com

4U Server Chassis

Logic Case 4U rackmount chassis are perfect for storage-dense applications, or where a lot of storage drives are to be installed, either in a fixed or hot-swap format. Our 4U chassis come in a diverse range of depths and feature-sets, including super short depth, all the way through to full...

www.logic-case.com

The LogicCases (at least the 36bay version) looks similar to a Fantec SRC-4240X07, probably with a different backplane and a 12 additional HDD bays.
So bascially, the first thing i would like to get out of the way is the case itself and as this is the position i am a little bit unsure about, maybe your opinions can help me out a little bit.

As usual, thanks in advance for your help and opinions. :)

 

[Ericloewe]

I can get the SuperMicro CSE-846TQ with a BPN-SAS-846TQ passthrough backplane for a reasonable price.

You realize you'd need 24 individual connections? Those backplanes are cheap for a reason. Maybe upgrading to an expander backplane is actually cheaper, too, since you'll need an expander (or multiple controllers) anyway.

 

[Fireball81]

@Ericloewe

Hello Eric.
Yea, i am aware. Those backplanes will be a pain to cable manage for sure but on the other hand, its a simple pass through backplanes, so i shouldn`t need to worry about any compatibility issues, different story with expander backplanes, at least with the cheaper ones. (judging from my experience) I have a bunch of controller lying around so that wouldn`t be a big issue but i see your point.

Usually i purchased 4U cases with expander backplanes from xcase.co.uk but they are sold out for quite some time now.
Any ideas for a 4U case with at least 24 drive bays and an expander backplane that works fine with TrueNAS and current, high capacity drives?

 

[NugentS]

I have two Logic Case cases, but I can't remember the model numbers (neither of the ones above). They are OK but cheap because:
1. Cheap backplanes
2. No PSU (and dual PSU's are not cheap)
3. Variable quality

Given this is for commercial, rather than home I think its worth spending on a better case designed for this sort of thing rather than having to add lots of expanders / HBA's. I had a real problem routing the cables for the last Logic Case I bought.
Also, the first case I bought (24 Port) was OK, but ATX only, rails were fine
The second I bought (24 Port) EATX, the rails were atrocious (not fit for purpose) and there were cabling issues through the fanwall - so I put the case on a shelf instead. I would have sent it back but I was kinda commited at that point.

The second case I have is I think the SC-4324

 

[Fireball81]

@NugentS

Thank you. Thats some valueable information. I was hoping for some decent quality with the Logic Cases.
I would like to go for a SuperMicro case but they use High RPM 80mm FANs on many of their 24bay/36bay storage enclosures and those can be nasty in terms of noise level.

Given the higher price of the SM cases, i might as well go with a full SuperMicro Server system, because when you add all the components i would need to complete the build, i get close to the price of the pre-build system anyways.

Among a few others i have this one here in mind:

SSG-540P-E1CTR36L | 4U | SuperServer | Products | Supermicro

4U UP front/rear-loading storage server with 36 hot-swap 3.5" SAS/SATA bays and PCIe 4.0

www.supermicro.com

 

[NugentS]

@Fireball81 the problem with fans is that for lots of disks you NEED the airpressure to keep temps under control. 120mm fans get in the way (as with my newer case)

I should add that the first case is absolutely fine from a cabling PoV and rails etc.
But the second, the rails (the only ones offered by the vendor) were too short and in the wrong place. Cabling the HBA to SAS Backplane cables was an utter nightmare and I am not convinced I won't have problems after a while - I might have to get a hacksaw / dremel out.

Well worth a "proper" solution so you can fit and forget in a business environment

BTW - I always sellotape a motherboard diagram to the top of the case so I know where things are.

 

[jgreco]

High RPM 80mm FANs on many of their 24bay/36bay storage enclosures and those can be nasty in terms of noise level.

No, the fans are not particularly high RPM, but they are noisy.

Noise is a natural side effect of the job they must perform. In order to cool your drives, air has to be forced between the little sub-1mm gaps between hard drives and chassis, where it does not want to flow willingly and easily. The fans are performing real work, and unlike the gamer or desktop PC's you may be more familiar with, heat buildup from 24-36 drives is a significant issue, and you do not want your server to turn into a modern EZ-Bake Oven.

If you operate your server in a cool environment, you may be able to get away with setting your fans to a lower speed in the IPMI, which can result in some lessening of the noise, while still allowing the IPMI to ramp them up if heat builds up.

The other alternative is to NOT fill every other drive bay, making a checkerboard pattern out of the used drive slots. If you do this, it is reasonably safe to replace the "noisy" fans with quiet fans. However, note that most such fans are "gamer grade" which means that they're designed to fail after just a few years, while the Supermicro industrial grade fans can usually run ten or fifteen years at full speed without failing. It is also super-important to make sure that you make it clear, perhaps with signage or labels, that it is NOT okay to fill the remaining bays with drives, or else you will get the aforementioned EZ-Bake Oven.

 

[Fireball81]

I should add that the first case is absolutely fine from a cabling PoV and rails etc.

Your first case was the SC-4824?

The fans are performing real work, and unlike the gamer or desktop PC's you may be more familiar with, heat buildup from 24-36 drives is a significant issue, and you do not want your server to turn into a modern EZ-Bake Oven.

Surely not. All of our servers are inside an air-conditioned server room. The "checkerboard pattern" would be an option, but i need the density, especially when we go with 24 bays. Changing the FAN profile from Optimized / High Speed to Low in the IPMI wasn't enough airflow to cool the 24 HDDs in our existing TrueNAS server, but of course with different FANs in a new system that might turn out differently.

I will have to check where i land with the DIY server and compare that to the aforementioned SSG-540P-E1CTR36L but if i go prebuild, that probably my option. Either that or the SSG-540P-E1CTR45L, but a top-loader will not do me any favors when it comes to noise levels, that's why i am a little bit hesitant to seriously consider that.

 

[jgreco]

Changing the FAN profile from Optimized / High Speed to Low in the IPMI wasn't enough airflow to cool the 24 HDDs in our existing TrueNAS server,

Well, good, then I don't need to hammer home the point too much.

but of course with different FANs in a new system that might turn out differently.

Well, I have like 15-20 years of experience with this 4U-24x3.5 form factor, and I can tell you that there is not any "turn out differently". The problem is actually inherent to the form factor, there just isn't enough space around the drives to provide good cooling without some significant energy spent on cooling, which turns into noise.

The available options basically boil down to finding a less dense chassis, which has gotten progressively harder over the years, or to "hack the chassis" in a clever way such as what I suggested, allowing more air to flow by lowering the drive density.

All of our servers are inside an air-conditioned server room.

Seriously, then, consider adding a new rack if you're out of space in your existing ones, and just go with a second chassis when you need to expand. You can definitely make that VERY quiet.

 

[Scharbag]

My 80mm/120mm Noctua fans at full speed in my Norco cases are not loud at all. Drives stay at reasonable temps but could be better. Mostly, this is due to my closet getting hot in the summer. Each of my Norco cases have 3 120mm and 2 80mm fans.

SAS Expanders are wonderful. A 9211-8i and an Intel RES2SV240 has enough lanes to connect to 24 drives in a case. $50 for the LSI card, $120 for the Intel card and you are done. Or you can get a proper Supermicro case with a SAS expander backplane and you just need a single HBA (these servers can be loud but why care if it is in a server room?). So many options out there and used Supermicro cases can be found pretty cheap. If you use an RES2CV360 expander, then you can have 24 drives in one case and then expand to another case in the future all while using only a single HBA (new case will need a new expander). All this old SAS2 stuff is dirt cheap now and plenty fast for a backup server (6Gbps channels). If you need SAS3, you will need to spend more (unlikely needed unless you are building a really large SSD system). But, you can also get SAS3 HBAs that have 24 lanes so a 24 disk enclosure needs only a single HBA and no expanders.

Anyway, there are options.

Cheers,

 

[NugentS]

"Your first case was the SC-4824?"

I don't know - and can't find the receipt

 

[Fireball81]

Seriously, then, consider adding a new rack if you're out of space in your existing ones, and just go with a second chassis when you need to expand. You can definitely make that VERY quiet.

Unfortunately, thats not a possibility. There are 10U left in the rack and i can`t fit another one in the server room. (its a very small room to begin with)

@Scharbag.
Thanks for the feedback.

these servers can be loud but why care if it is in a server room?

Well, thats the thing. We have a 2nd Editing room exactly near the server room and you can already hear the noise. (with components in there that are pretty mild in terms of overall noise level) Adding another SuperMicro server with heavy duty FANs might be a little bit much for my colleague working in there.

As we need capacity over speed, we will use 6Gb/s SATA harddrives. (probably Seagate Exos 18TB) In other words, SAS2 will surely suffice.
There are a lot of options indeed. Serverschmeide.com for example offers a SM 847 CTO, that could be an option too.

Serverschmiede.com GmbH

Vertrieb & Konzeption von refurbished Enterprise Server & Storagelösungen im Business Umfeld - maßgeschneiderte, wiederaufgearbeitete Server & mehr

www.serverschmiede.com

@NugentS

Thanks anyways. :)

 

[NugentS]

Bet thats loud - its gonna need a lot of air to keep 36 drives cool. Otherwise looks very good - nice price too