BUILD External SAS Raid Controller + SAS Expander

[Mehul V]

Hi. My first time on the form, first thread and first question, so please forgive me if I do something wrong.
So far, I have read the following threads regarding my problem:

More Useful:

• https://forums.freenas.org/index.php?threads/external-sas-controller-selection.36930/
  
• https://forums.freenas.org/index.php?threads/sas-expanders.36394/

Not so useful in my case:

• https://forums.freenas.org/index.php?threads/sas-controller-with-external-sf-8088.21489/
  
• https://forums.freenas.org/index.php?threads/extending-storage-with-external-raid-case-zfs.1358/

My problem:
I am looking for a SAS Raid Controller with atleast 2 external SAS ports and maybe atleast 2 internal SAS ports. This controller must be compatible with FreeNAS (DOH!) and also be able to run in IT mode (I don't mind flashing it to IT, if every other requirements are met).

I currently have:

• HP 24 Bay 3Gb SAS Expander Card (hp.com) - Option part number - 468406-B21
• LSI MegaRAID SATA/SAS 8888ELP 3Gb/s PCI-Express w/ 512MB onboard memory RAID Controller Card, Kit (Newegg.com)
• HP Smart Array P421/1GB FBWC 6Gb 2-ports Ext SAS Controller
• LSI SAS/SATA 9750-8i PCI-E x8 6Gbps RAID Controller

What I am still looking for:

• FreeNAS Compatible SAS Raid Controller with 2 external and 2 internal port
• SAS Expander (If the above mentioned controller doesn't work with the Expander that I already own)

Let me know if you need any more information from my side.

Thanks!

EDIT 1: Added HP Smart Array P421 and LSI 9750-8i

 

[Mirfster]

Cursory look at those two cards you already have indicates that they won't work for FreeNAS. Of if they do it won't work too great. They are older models (3Gbs) and the HP Card says:

• Up to 14TB of total storage with 14 x 1TB 3.5-inch SATA MDL HDD
• Up to of 7.2TB total storage with 24 x 300GB 3G SAS 10K SFF DP ENT HDD

Which makes me believe that it may not be able to see drives over 2TB.

Are you basically looking to expand drives since you are running out of room? If so, not sure what system specs you have but maybe if you told us we could provide alternate suggestions.

 

[Mehul V]

Cursory look at those two cards you already have indicates that they won't work for FreeNAS. Of if they do it won't work too great. They are older models (3Gbs) and the HP Card says:


Which makes me believe that it may not be able to see drives over 2TB.

Are you basically looking to expand drives since you are running out of room? If so, not sure what system specs you have but maybe if you told us we could provide alternate suggestions.

Sorry, I should have made it clear. The two things I already have can totally be ignored. They were just to play around with and getting familiar with SAS technology and the controller/expander cards were to understand how it all connects. I have not and neither do I plan to use them for any of my FreeNAS builds.

I am starting from scratch. I already have a motherboard (X8DTI-F) and 2 x E5620 with 76GB DDR3 1333 ECC RAM. I am also looking at 10 x 4 TB Seagate Barraucudas for the storage. I *MAY* also looking into making another storage LUN or cluster with just SSDs for flash storage, but I don't know anything about flash storage clusters at this moment.

Sorry about the confusion =/

 

[Mirfster]

I am starting from scratch. I already have a motherboard (X8DTI-F) and 2 x E5620 with 76GB DDR3 1333 ECC RAM. I am also looking at 10 x 4 TB Seagate Barraucudas for the storage. I *MAY* also looking into making another storage LUN or cluster with just SSDs for flash storage, but I don't know anything about flash storage clusters at this moment.

In that case you may want to consider a 2U SuperMicro case (If you don't mind a 2U). Since you want 10 (or more drives), the other contributors here can provide suggestions on a viable case that would suit your needs.

Sorry, I don't run SuperMicro so would not be the best source on that matter. If others don't chime in on this thread, then perhaps start a new one along the topic of "Looking for a case". Add what you already have and your desires and I am sure you will get some input.

 

[Mehul V]

In that case you may want to consider a 2U SuperMicro case (If you don't mind a 2U). Since you want 10 (or more drives), the other contributors here can provide suggestions on a viable case that would suit your needs.

Sorry, I don't run SuperMicro so would not be the best source on that matter. If others don't chime in on this thread, then perhaps start a new one along the topic of "Looking for a case". Add what you already have and your desires and I am sure you will get some input.

Thanks for the suggestion.
I don't mind the 2U case, but I am currently only looking for recommendations regarding SAS Controller and SAS Expander.

 

[Mirfster]

For SAS Controllers, usual recommendations are:

• LSI 9211-8i
• IBM M1015 (Cross-Flashed)
• Dell Perc H200 (Cross-Flashed)
  • These are what I mainly use
• Dell Perc H310 (Cross-Flashed)

For SAS Expanders, that is usually on the BackPlane of 2U Cases. Otherwise, for connecting External Units (JBODs); then I personally use LSI 9200-8e and LSI 9201-16E.

For something that has both Internal and External, all that comes to mind is the LSI 9207-41-4e. There may be other options, just not too sure.

 

[Mehul V]

For SAS Controllers, usual recommendations are:

• LSI 9211-8i
• IBM M1015 (Cross-Flashed)
• Dell Perc H200 (Cross-Flashed)
  • These are what I mainly use
• Dell Perc H310 (Cross-Flashed)

For SAS Expanders, that is usually on the BackPlane of 2U Cases. Otherwise, for connecting External Units (JBODs); then I personally use LSI 9200-8e and LSI 9201-16E.

For something that has both Internal and External, all that comes to mind is the LSI 9207-41-4e. There may be other options, just not too sure.

Thanks! I like both LSI 9200-8e and LSI 9201-16E.
If I can't find a decent one that has 2 Internal and 2 External SAS ports, I might just have to use the external cards you mentioned here.

 

[Mirfster]

Just for the record if the system has a true SAS Expander Backplane, then you would only need one Mini-SAS to Mini-SAS cable to connect the HBA and BackPlane. This would allow you to see all the drives from one connection.

If it does not have a Expander Backplane, then you would end up using something like a Mini-SAS to SATA Breakout Cable. Which would only let you see 4 disks per connection.

Some older models have a backplane that has SFF-8484 connectors. Which (if I recall correctly) require one connection for each set of drives. Think of it as if the chassis holds 12 drives and the backplane has 3 SFF-8484 connectors. Each one then controls 4 drives, so in the end you would need three Mini-SAS (SFF-8087) to SFF-8484 cables.

Examples:
Mini-SAS to Mini-SAS (SFF-8087 to SFF-8087)
Mini-SAS to SATA Breakout (SFF-8087 to 4 SATA) - Note you would want a "Forward" Breakout
Mini-SAS (SFF-8087) to SFF-8484

Real world example:
I run all Dell C2100/FS12-TY Servers. They each hold 12 (3.5" or 2.5") Drives and 2 x 2.5" internally (I also have the model that hold 24 x 2.5" and 2 x 2.5" internally).

My Main System

• One Perc H200 (Mezzanine Model) HBA
  
  • One SFF-8087 to SFF-8087 cable connected to the Backplane (SAS Expander); which houses 12 Drives. Even though the backplane has two SFF-8087, I only need one connected. I could connect both for a "Wide Channel", but lets skip that for now..
    
  • One SFF-8087 to SAT Breakout; which I can use to connect up to 4 SSDs internally. *** Can mod the system to hold more SSDs internally if desired.
• One LSI 9200-8e (soon to be replaced with a LSI 9201-16e for more external ports)
  
  • This connects to another Server that acts as a JBOD
  • Connected via a Single SFF-8088 to SFF-8088 cable
  • Again I could do two connection to form a "Wide Channel", but have not needed it thus far

JBOD System

• One Dual Mini SAS 26 Pin SFF-8088 to SAS 36 Pin SFF-8087
  • One SFF-8087 to SFF-8087 cable connected to the Backplane (SAS Expander); which houses 12 Drives.

So, if I wanted to I could attach another JBOD and have additional drives as well. I prefer to separate the Volumes/Pools so that each JBOD is "Self-Contained". That way I eliminate dependencies between the boxes.

I have changed stuff (and plan on changing more); but here is a thread where I originally mentioned the setup.

Hope this helps.

 

[Stux]

It is possible to have SFF-8087 -> SFF-8087, without an expander. For example, the Norco chassis use a passive backplane, which essentially is a 4x breakout, but connects with a straight-through SFF-8087 cable.

 

[Mehul V]

Just for the record if the system has a true SAS Expander Backplane, then you would only need one Mini-SAS to Mini-SAS cable to connect the HBA and BackPlane. This would allow you to see all the drives from one connection.

...

Hope this helps.

Really appreciate you taking the time to mention what you have currently setup and also the different options I can use.
I also didn't know about the HBA to backplane connection options.

I decided to pickup 4U 24bay server with 3 x LSI 9211-8i:
I dont even have 24 drives right now to begin with, so I dont think I will be needing to add external storage to this server anytime soon. This eliminates the need of getting a new expander and a raid controller with external SAS ports.

With a total of 6 x SFF-8087 mini-SAS, I am assuming they should connect directly to the backplane, but will know once the server arrives.

Here are some specs of the server, in case anyone is interested:

CPU: 2 x Intel Xeon CPU E5-2650 0 @ 2.00GHz
RAM: 64GB ECC DDR3 (Might just use 32GB depending on how much hard drive I stuff in the case)
Raid Cards:3 x LSI 9211-8i
Hard Drives:
5 x 4TB (ST4000DM000) I might use more, not sure yet
5 x 120GB SSD(SSDSC2BW120A4K5)
3 x 240GB SSD(SV300S37A/240G)