SOLVED Supermicro X10DRH-CLN4 - Broadcom 3008 + Intel SAS expander RES3TV360

[boatymcboatface]

Greetings,

I have a question I'm hoping someone can help answer.

I own a Supermicro X10DRH-CLN4 motherboard which comes with 2 Mini SAS HD connectors powered by a broadcom 3008 SAS chip.
At present, I have 2 M1015 cards plugged into that motherboard with cables going to a backplane using (SFF‑8087 x SFF-8087) cables for a total of 16 drives.

I can potentially get my hands on an Intel RES3TV360 SAS3 expander on the cheap. Seeing how I expect to grow the amount of drives and dabble with SAS3 stuff in the future, I was wondering if for now I could consider selling off my M1015s, putting the onboard 2 x 3008 Mini SAS HD in the expander, and then connecting that using mini SAS HD to SFF-8087 cables to my backplane, which would allow me to have some future room for SAS3 connections on the remainder of the unused expander ports. I realize it would be capped at 6GB for now due to my backplane, but I was thinking the rest of the ports could drive a 12GB enclosure.
Would that work? How many drives can I connect without performance loss using the 2 onboard connectors?

Many thanks,

BmBf

 

[Johnnie Black]

With a dual link to the expander you'll have 4400MB/s max usable bandwidth with SAS2/SATA3 devices, a little under double that (limited by the PCIe 3.0 x8 link) if you upgrade them all to SAS3.

 

[boatymcboatface]

Sorry I'm a bit confused. If it's 4400 max bandwidth on devices that are capped at 6GB, and double-ish that at 12GB, wouldn't a single 12GB uplink then be enough to connect to the expander to provide the 4400? I mean, the expander itself is 12GB like the uplink, so I was assuming only the individual ports that you hooked up to 6GB backplane ports would run at half speed, the expander itself would get the double rate. Meaning that if I used two uplinks, the expander would be able to sustain 8800ish on all ports combined, but each 6GB port would simply take less of that total as it's at max 6GB rate? Or does it limit the total speed to 4400 as soon as one of the ports runs at 6GB speed?

I guess what I'm trying to say is that if the expander is fed 8800, and I attach, say, 4 SATA hdd's to each 6GB port that would eat up max 250MB/sec per disk - I would need 1000MB/sec per port, and I could use 4 disks on 8 ports to get close to the theoretical 8800. Is that true, or is my logic painfully flawed? :)

Thanks!

BmBf

 

[Johnnie Black]

You're confusing gigabits with gigabytes, each 4 lane link has 4 x 600MB/s for SAS2, 4 x 1200MB/s for SAS3, so dual link gives you 4800MB/s and 9600MB/s max theoretical bandwidth respectively, there's always some overhead, and in my experience each SAS2 link will give 2200MB/s usable (of the 2400MB/s theoretical max)

 

[boatymcboatface]

Thank you. Does that mean then that indeed the expander would be supplied with 9600MB/s max, since it's a SAS3 expander with two links, and then you can just "deduct" the speed of individual drives from that up to the limit? As in it could drive 24 SATA harddisks without saturating? I understand each SAS2 port with 4 lanes would take 2200MB/s max but since 4 SATA drives on that link would not hit that bandwidth, and there are more ports to use on the expander, I'm assuming it could do the 24 SATA drives without issue this way?

Many thanks for your help, and sorry for being a bit thick, am rather new to the SAS concept :)

 

[Johnnie Black]

Does that mean then that indeed the expander would be supplied with 9600MB/s max, since it's a SAS3 expander with two links, and then you can just "deduct" the speed of individual drives from that up to the limit?

No the link speed is the result of the devices used, only with SAS3 devices can you get 9600MB/s, and that's with all SAS3 devices, if you also have SAS2 devices connected and they are all accessed simultaneously it will bring the speed down.

I understand each SAS2 port with 4 lanes would take 2200MB/s max but since 4 SATA drives on that link would not hit that bandwidth, and there are more ports to use on the expander, I'm assuming it could do the 24 SATA drives without issue this way?

Yes, say you have a dual link to the expander and are using all SATA3 drives, you'll have 4400MB/s total the 24 disks, ~180MB/s for each when they are accessed simultaneously.

 

[boatymcboatface]

Ah, that answers all my questions. Thank you very much for your feedback, much appreciated. Obrigado!