Where to put non-SAS drives in a server chassis with SAS bays?

[kuloch]

I'm building a NAS server and would appreciate clarification on a couple seemingly vague or potentially controversial details that leave me a bit confused after a couple months of frequently reading up on various guides, common mistakes, build recommendations, etc. I'll split the questions into separate threads to keep each discussion limited in scope.

Where are the non-SAS drives typically put in a server chassis with many bays that are all on a SAS backplane?

I'm aware that an SATA drive can be used on a SAS backplane, but I'm specifically talking about wanting to also use the motherboard's onboard SATA ports.

For example, I'm considering a Supermicro CSE-847 (24 front bays, 12 rear, all on backplanes). Perhaps it's uncommon to use the motherboard's SATA ports in such a chassis (preferring M.2 or PCIe slots for any non-SAS drives?), but I can't find a good recommendation on where to put the SATA drives. The only option (using above chassis as example) seems to be yanking out the rear backplane and running SATA cables to those bays from the motherboard. But this feels like a kludge where I'm likely missing a better way. I've built my own PCs for many years, but SAS (and enterprise-grade in general) was brand new to me 3 months ago (and yes, I've read many primers and guides since).

My preference is to have a decent few ports available for SAS drives and still be able to use the onboard SATA ports, such as for an array of mirrored SSDs (where the SAS drives will be for a raidz2 pool). If it's relevant, I've already purchased a Supermicro X12SCA-5F and Intel Xeon W-1350 (and yes, I'm aware this is only supported by last week's Bluefin release).

Or, given my example chassis above (and depending on my other question):
Would I likely be better off using the rear SAS backplane for SATA SSD drives and generally ignoring the onboard SATA ports (or intentionally underutilizing them)?

 

[Ericloewe]

Simple answer: You don't, it's not practical outside of some edge cases - rear 2.5" bays in Supemicro chassis, for instance. Maybe if you had 12 SATA ports on the motherboard (which you don't, really) the rear backplane could benefit from some cost savings, but that's a fantasy scenario.

The SAS controller(s) already support(s) all the available disk bays, so this is a non-issue. And why would you want to use SAS disks? 99% of the time, it's extra cost for no meaningful extra features.

Would I likely be better off using the rear SAS backplane for SATA SSD drives and generally ignoring the onboard SATA ports (or intentionally underutilizing them)?

Yes, unequivocally.

 

[kuloch]

Hmm, it doesn't look like the forum allows me to edit my post to add a link to my other question about using SATA behind a SAS expander - likely a new user restriction.

I'm guessing by the "why would you want to use SAS disks?" question you're in the camp of having no concerns with running SATA drives off a SAS expander backplane. I've read quite a few issues and concerns along those lines, but I'll leave that discussion to the other thread.

Related to this thread is the follow-up question:
Where do I put my TrueNAS boot drive?
It's a cheap SATA SSD with 120 GB. I thought I read something about being unable to boot off drives attached to a SAS expander. Or maybe on an HBA at all. Or maybe I'm mixing things up from cramming too much information in such a short time (in addition to working on certifications for work during the same). Will this SATA SSD boot just fine on a SAS expander (through something like an LSI 9207-8i)? Or does it need to be directly plugged in with SATA cable to a motherboard port? (or alternatively attached via SATA-to-USB adapter, which I'd planned to do if eventually needing to use all the SATA ports)

 

[jgreco]

Where do I put my TrueNAS boot drive?
It's a cheap SATA SSD with 120 GB. I thought I read something about being unable to boot off drives attached to a SAS expander. Or maybe on an HBA at all. Or maybe I'm mixing things up from cramming too much information in such a short time (in addition to working on certifications for work during the same). Will this SATA SSD boot just fine on a SAS expander (through something like an LSI 9207-8i)? Or does it need to be directly plugged in with SATA cable to a motherboard port? (or alternatively attached via SATA-to-USB adapter, which I'd planned to do if eventually needing to use all the SATA ports)

Naw, that's stupid. Some people refuse to flash the BIOS onto the HBA, which renders it unbootable, but I think that's really dumb because it makes it super difficult to debug the setup outside the UNIX environment. A modern SATA SSD drive should be just fine on a SAS expander, booting or whatever. And they're cheap enough that if you inadvertently use a 3Gbps SATA SSD, the main thing I'd expect MAYBE might not work, you can easily replace it.

Also see

Highly Available Boot Pool Strategy

I've pounded a few versions of this out over the years, but I hate explaining over and over. TrueNAS allows a ZFS boot mirror pool to be created to increase the reliability of the NAS. This sounds great in theory, but there's a flaw. Due to...

www.truenas.com

 

[Ericloewe]

Some people refuse to flash the BIOS onto the HBA, which renders it unbootable, but I think that's really dumb because it makes it super difficult to debug the setup outside the UNIX environment.

I'm sort of in that camp, but my position is more nuanced:

• Not booting from the card? Consider not flashing the extension ROM.
• Multiple cards? Flash the extension ROM only to one card - or now that the firmware is more or less stable (for better and for worse), lag exactly the same version to all cards.
• Only flash the legacy BIOS OPROM if absolutely needed for BIOS boot.

The extension ROM adds little value if not booting from the card. Conflicting expansion ROMs have been known to cause bizarre issues at boot time. The UEFI extension ROM seems a bit more sane than the old BIOS extension ROM.

 

[danb35]

Where do I put my TrueNAS boot drive?

Under the motherboard tray, cabled to one of your motherboard's SATA ports. Unless you expect an awful lot of shock and vibration, you can pretty safely let a SSD just hang by its cables, but if you want a bracket, Supermicro makes one. Here's a vendor on eBay that looks like they have the right thing (I hope so; I just ordered one):

SuperMicro MCP-220-84701-0N Internal drive bay for one 3.5" or two 2.5" HDD | eBay

Find many great new & used options and get the best deals for SuperMicro MCP-220-84701-0N Internal drive bay for one 3.5" or two 2.5" HDD at the best online prices at eBay! Free shipping for many products!

www.ebay.com

Or, if you can whack in one of those 2x2.5" bay hotswap cages in the rear of your chassis, in there--but still wired to the motherboard SATA ports.