SAS2-846EL1 Backplane Connection(s)

[malcolmputer]

I just purchased myself a fancy SC846E16-R1200B with the SAS2-846EL1.

I have a M1015 and a couple of SFF-8087 cables.

In the SAS2-846EL manual page 2-1 it lists all of the SFF ports, and I have the EL1 model which has ports 7, 8, and 9. I read through the manual and it never tells you why there are three connections. 7 is always shown connected to the HBA and 8 is show in cascade configurations, but 9 is never used. Does 9 have a purpose?

I guess my question is, with a M1015 should I just run one cable to 7? Is there some way to utilize the extra four SAS2 lanes, or am I stuck with only 24Gbit?

I don't have another chassis for cascade, and I am using SATA discs so I don't care about fail-over etc.

 

[jgreco]

There are three connections because you have 24 connections used for drives on the backplane, and it is an LSI 2x36 port SAS expander which has 36 PHYs. After subtracting 24 PHY's for the drives that works out to 12 remaining ports, of which four are on each SFF8087.

You can do whatever you'd like. In addition to what is suggested by SuperMicro, you could take a SFF8087 breakout cable and run that to internal 2.5" disks (the 846 IIRC has a two drive bracket option while the 846B has an internal bracket PLUS the two rear 2.5" bay option), etc.

In theory you should be able to SAS wideport eight lanes. In practice, I tried it some time ago and ran into issues, mostly that performance wasn't good (worse than a normal x4). I didn't have any reason or motivation to care too much since 24Gbps is still quite reasonable to an array. Please do feel free to experiment. I'm kinda waiting for someone to tell me in what manner I was an idiot. ;-)

 

[malcolmputer]

In theory you should be able to SAS wideport eight lanes. In practice, I tried it some time ago and ran into issues, mostly that performance wasn't good (worse than a normal x4). I didn't have any reason or motivation to care too much since 24Gbps is still quite reasonable to an array. Please do feel free to experiment. I'm kinda waiting for someone to tell me in what manner I was an idiot. ;-)

So, I *might* be able to run a SAS cable from SAS0 on the M1015 to 7 and then run another SAS cable from SAS1 to 8 or 9 (probably 9) and in theory it would connect 48 Gbit to the 24 drives?

What would be the best way to benchmark it / checkup on link speeds (theoretical and practical) from the M1015 in FreeNAS? I am really green as far as SAS is concerned. I assume something like camcontrol devlist isn't going to tell me the negotiated speeds. I could try dd if=/dev/da{x-n} of=/dev/null to get read speeds, but I am not 100% sure I would break the 24GBit barrier.

 

[Darren Myers]

@DataKeeper and i both use the same chassis as you, and we both have SAS cables connect to ports 7 and 8 to our M1015's.

 

[malcolmputer]

@DataKeeper and i both use the same chassis as you, and we both have SAS cables connect to ports 7 and 8 to our M1015's.

That makes me feel better. Any problems with it? What kind of speeds do you get?

 

[Darren Myers]

That makes me feel better. Any problems with it? What kind of speeds do you get?

Problems, not as of right now. General problems, none thus far. The biggest complaint i see from people is its "loud" or "too loud".....i get really good speeds i feel, i run 3 RAIDz3's and i can write almost ~400Mbps constant or so, depending where its going to/from

 

[jgreco]

Well, the real question is what's the performance like to the individual components.

 

[malcolmputer]

Well, the real question is what's the performance like to the individual components.

So far, scrub speeds are the same with both connections and with one connection. I have two pools with 2x6 in z2 in one and 1x6 in z2 in the other and I am getting around 600 MB/s total from both.

I am not CPU bound, shouldn't I be getting more than 600MB/s total from 18 disk in three 6 disk Z2s?

Once the scrubs finish I will try dd-ing reads from all disks simultaneously and see if I can see a speed difference between two cables and one cables there.

Also, strangely enough, the LSI bios config screen shows each disk at 3Gbit SATA link speeds except in a set of half empty bays where it shows 6Gbit for three disks. I know that I don't have that much bandwidth with even two full M1015 links, so I assume it is lying. (unless it is reporting the max it could pull from that disk while the others were idle, which would be more sane I guess).

 

[jgreco]

The reported link speed is just the negotiated speed, not what it's actually capable of when competing with a bunch of other drives on an oversubscribed SFF8087.