Multipath Considerations

[insolent]

Hello,

I'm looking to expand my storage, and I was considering using external enclosures. Is multipath a sound option, or am I walking into a world of... chocolate ice cream emojis?
Are there specific gotchas to watch out for when choosing my hardware, jbod chassis, hba cards?
Are there known good practices to adhere to for the sake of performance/reliability/economy?

Is it preferred to path to/from the same card, or a different card?

My intent is to use SAS drives, but I'm curious how I would get away with SATA. I would presume (I know I am using the wrong terms) that I would need to isolate sata drives to specific channels on the expander.

I'm happy to try things if folks simply don't know.

 

[jgreco]

You cannot do multipath with SATA unless you use interposers.

SAS has a primary and secondary communications channel structure, and "multipath" uses both of these. You are creating two separate-but-parallel topologies to attach your disks. Unfortunately, SATA disks lack the secondary port, so there is no good way to do this with them.

You typically multipath to a separate controller card to avoid loss of connectivity due to a failed HBA. It does happen.

Multipath is generally a lot of work to set up and maintain, plus slight extra operational costs due to the power consumed by the secondary channel HBA and SAS expanders.

 

[Ericloewe]

That said... You can go with the external enclosures and just forget about the multipath stuff.

In my opinion, multipath on a single host is a bit of a solution looking for a problem. There's still a ton of single points of failure. A CPU goes bad? Server is down. A DIMM develops an uncorrectable error? Server is down. BMC freaks out? The server might not be down, but your next downtime might be longer than expected.
So, you'd probably want to either go the simple route or the full High-Availability route (which naturally costs more). The in-between route is marginally better on paper, but feels irrelevant in practice.

 

[jgreco]

Eh, it can save your arse when a cable gets knocked loose.