Advice on a few cheap "shelves" to consider

[David Dyer-Bennet]

That is, external boxes that hold a batch of disks and connect to the server, to expand capacity. Particularly, the kind that have an SAS expander rather than their own controller. (If other terminology would be clearer you could enlighten me on that, too!)

My understanding is that, with the right kind of "shelf" of this sort, it's quite easy to take an SAS connection from something like an IBM M1015 (or an LSI 9211-8i) and run a cable out somehow, or use a slightly different card variant that has an external port, and run a bunch of SATA disks (I'm thinking the 12-24 range) outside the server chassis.

However, I've never done this before. I've got an add-in SAS controller and the cables to let it drive 8 SATA drives directly, so I'm not completely unfamiliar with that technology, but I'm replacing it (it's an old one, limited to 2TB drives, and none of my current systems are using drives that small). Searching the web, and Ebay, for "shelves" finds me so many different things at such range of price points that I find I haven't the faintest clue what might actually work.

I'm guessing there are people here who know off the top of their heads some things that are among the most cost-effective current choices. Could somebody give me some pointers on exact models to consider, and any easy-to-make mistakes to avoid? I'm definitely needing SATA disks as the storage element -- performance is entirely adequate in my experience so far (on my workloads, I mean) and price is a very important consideration.

(I've read "Confused about that LSI card?" here, and other things that I could find that looked relevant.)

(Wow; an LSI 9201-16e sounds terrifying for driving far more shelves than I'll ever need, containing more disks than I can possibly afford, and is cheap on ebay right now. It's overkill, but it's half the price of the 9211-8i right now; is there some reason it won't work with FreeNAS?)

(I do understand that spreading a zpool across shelves makes it easy for one failure (cable, power, power supply, or whatever) to damage the pool beyond repair. Don't plan to go crazy in this direction, but my early investigation involves pushing at least mentally at the limits, to understand them. Separate zpools on each shelf, if it fits the overall needs, avoids that set of problems though.)

New shelves that look okay, like this, cost $700 (without drives of course) for 8 drive bays; for $700 I can build TWO 10-drive servers (not including drives) from scratch, so it's not cost-effective.

 

[depasseg]

It's so simple that once you do it, you'll be wondering why you haven't done it already.

I hooked up a SuperMicro SC847E16 with 45 drives using a single external SAS cable.

 

[David Dyer-Bennet]

I do need to identify a lower-spec chassis, though (well, lower price, if possible; but I don't need 45 drives, either, so perhaps there is hope). That price has extra digits in it .

Hey, I see you confirming the 9200-8e works with FreeNAS, though!

 

[depasseg]

I got this on ebay for $1100, and I'm sure there are lower spec models for less. If you have the room, I'd recommend getting more than you need. I figured this way, I found a deal on 20-ish 2TB drives so I use that as backup pool and I also have a bunch of empty drive slots to expand my primary pool if I need to.

 

[Dice]

hello,

Beyond what's already mentioned, I'd suggest you revisit the hardware guide by @cyberjock. There are some valuable pointers there regarding choices on chassis. Most notably the idea of mishmashing your own mix of '5in4 hotswap addins', consumer grade PSU's and poor cooling ..which ends up costing about the same as a vastly superior chassis designed and built to the task. Basically it boils down to his rule #4 in that guides intro. It will bite you in the arse sooner or later if aiming too low. (mine is already sore)
You can find the thread here: https://forums.freenas.org/index.php?threads/hardware-recommendations-read-this-first.23069/

I'll share my personal "Do's & Don't" on 'getting the first proper rack box/shelf for +24 drives, which is based off readings and recent personal experiences and a lot of trying to 'think ahead of need' while buying equipment that all line up together:

This is the better path:

• Get a chassis solution including PSU's designed to hold X drives.
• Make sure you get a SAS2 backplane, preferably with built-in expander
• Get a bigger box than you ever anticipate filling.

Avoid this path or lines of thinking:

• "I'll upgrade to a SAS2 backplane later" : Backplanes are expensive.
• Getting consumer grade PSU's hoping to run > 25 drives. Power might be there on the sticker, but doing the cabling right is a mess. Trying to juggle two PSU's, adding jumpers here n there... naaah..
• Thinking you could add an SAS expander later on. It'll just cost you more than getting a deal where it is included.
• Aiming for the cheapest new 24bay box, thinking you made a good deal. In reality the deal is probably terrible compared to second hand SM 846 or 847boxes when taking into account total costs as fans, psu, quality etc.
• Changing high power fans to slower consumer grade fans without being <really sure> on what you are doing. Risks of toasting drives and other components are not 'can happen' but highly likely to happen.

Feel free to comment on these lists too, I'm all ears.

Cheers /