Drive Replacement

[Davvo]

Likely any cable you can find on Amazon is guaranteed to work better than a detective one.
I am not clear which suggestion you are asking about. SATA to SATA? For how many drives?

 

[Etorix]

If there's any SATA Cable like above, it would be much great.

Anything(*) you may find by searching for "sff-8643 sata breakout" on Amazon/Newegg/eBay/your favourite reseller:

Amazon.com : sff-8643 breakout sata

For SATA drives, prefer shorter cables.

(*) If specified, go for "forward breadkout", NOT "reverse breakout".

 

[Fastline]

Likely any cable you can find on Amazon is guaranteed to work better than a detective one.
I am not clear which suggestion you are asking about. SATA to SATA? For how many drives?

Looking for SFF8643 to SATA like the following design:

I prefer this design as there is very nominal chance that the whole connector will be disturbed when i clean or something and my drives will not have CRC errors. The other systems i have, they have all GIGABYTE SATA to SATA Cables and even if its connected well, the CRC shit happens. I don't trust those cables anymore. Of course, its not the drives cause the one my friend gave, the error count did not increase.

 

[Fastline]

(*) If specified, go for "forward breadkout", NOT "reverse breakout".

Umm, is there any difference between the two?

 

[Etorix]

"Forward" does what you want: From a motherboard/HBA to four individual drives.
"Reverse" goes from four individual SATA ports on a motherboard to a SFF-8643 backplane connector. (Normally, one cannot end up with a reverse cable without really looking for it.)

 

[Fastline]

"Forward" does what you want: From a motherboard/HBA to four individual drives.
"Reverse" goes from four individual SATA ports on a motherboard to a SFF-8643 backplane connector. (Normally, one cannot end up with a reverse cable without really looking for it.)

Thanks for the info!

 

[Fastline]

Just wanted to ask something about the HBA Card. Pretty much everything has a relation with the lanes in a computer but is there any concern when utilizing 16 Drives on a LSI HBA (9400-16i)? Like will it run at full speed without any restrictions, each drive at 6Gb/s connection or is there any kind of limitation or something i shall be concerned about?

For the onboard SAS Port or the another same HBA Card, i would need to connect at least 8xU.2 NVMe Drives for fast storage. Any limitations on that side?

 

[Davvo]

Looking for SFF8643 to SATA like the following design:
View attachment 73471

I prefer this design as there is very nominal chance that the whole connector will be disturbed when i clean or something and my drives will not have CRC errors. The other systems i have, they have all GIGABYTE SATA to SATA Cables and even if its connected well, the CRC shit happens. I don't trust those cables anymore. Of course, its not the drives cause the one my friend gave, the error count did not increase.

SATA drives can safely be plugged into SAS connectors, but SAS drives cannot be plugged into SATA connectors.

Drive Replacement

SAS drives generally cost more than their SATA counterparts, have poor SATA reporting and require a SAS HBA. SAS backplanes and expanders are certainly useful, but I struggle to see any benefit of SAS for the hard drives themselves.

www.truenas.com

Just wanted to ask something about the HBA Card. Pretty much everything has a relation with the lanes in a computer but is there any concern when utilizing 16 Drives on a LSI HBA (9400-16i)? Like will it run at full speed without any restrictions, each drive at 6Gb/s connection or is there any kind of limitation or something i shall be concerned about?

For the onboard SAS Port or the another same HBA Card, i would need to connect at least 8xU.2 NVMe Drives for fast storage. Any limitations on that side?

If you are using HDDs there is no way you can saturate the HBA's bandwith with 16 drives if you are using a x16 slot.

 

[Fastline]

SATA drives can safely be plugged into SAS connectors, but SAS drives cannot be plugged into SATA connectors.

Drive Replacement

SAS drives generally cost more than their SATA counterparts, have poor SATA reporting and require a SAS HBA. SAS backplanes and expanders are certainly useful, but I struggle to see any benefit of SAS for the hard drives themselves.

www.truenas.com

Thanks to this forum, i'm aware of that ;)

If you are using HDDs there is no way you can saturate the HBA's bandwith with 16 drives if you are using a x16 slot.

Can you please rephrase it for me? I just want to know whether all my drives connected to the HBA Card will perform as expected or not. I mean all 16 drives connected into 4x SFF8643 SAS port on a single HBA Card which will be installed in an X16 Slot will be able to perform 6Gb/s each? As that's the one SATA Port Bandwidth for the onboard i guess. I'm not sure if 6Gb/s bandwidth is for one single SATA (onboard) port or combined together (for all the SATA ports present on the board).

I read somewhere in this forum regarding the lanes and HBA so I'm little confused. I don't want any of my drives which are connected on an HBA Card under performs after so much of investment in the hardware thingy.

 

[Davvo]

Thanks to this forum, i'm aware of that ;)

Then I am not understanding why you are asking for another recommendation xD

Can you please rephrase it for me? I just want to know whether all my drives connected to the HBA Card will perform as expected or not.

If we are talking about HDDs, and you are plugging the HBA into a full size PCIe slot (thus using all the HBA's available bandwith) yes; otherwise you need to do the math with the lanes.
HDDs don't come even close to saturating the SATA III bandwidth.

 

[Fastline]

I am not understanding why you are asking for another recommendation xD

Cause i need one. Will explain tomorrow :)

otherwise you need to do the math with the lanes.

Can you tell me more about this please?

 

[Davvo]

Cause i need one.

What's wrong with the one I originally suggested?

Can you tell me more about this please?

If you use the HBA into a smaller than required PCIe slot (such a x6 open slot) the available bandwith decreases.
Since SSDs are much faster than HDDs, they can easily saturate the available bandwidth; this can also happen with very big configurations that use cascaded HBAs.

 

[Fastline]

What's wrong with the one I originally suggested?

No, no nothing wrong with that. Those were bought already. That was SFF8643 to SAS. I'm looking for SFF8643 to SATA. As already explained, till now i was using onboard SATA to SATA (Gigabyte). Over the period of time, i see a lot of CRC errors even when it had a SATA lock and when i replaced the cables which my friend gave, the CRC error count did not increase. So, I'm looking for SFF8643 to SATA cables like the following design. I find it more safe and not prone to CRC shit as during the maintenance it has a very nominal chance for loose connection. Even when i ensured that it has no loose connection after maintenance, it had CRC errors. Those were brand new original cables which came with the Motherboard.

In short, SFF8643 to SATA but like the following design:

If you use the HBA into a smaller than required PCIe slot (such a x6 open slot) the available bandwith decreases.

Makes sense. So, if i install the HBA into a PCIe Slot which is not only X16 slot but also has X16 bandwidth, will my all 16 drives run at 6Gb/s each? LSI 9400-16i says its a 12Gb/s. Is the 12Gb/s bandwidth for the all SAS ports combined or is it 12Gb/s each SAS port? Sorry, just trying to figure out things here.

 

[Davvo]

No, no nothing wrong with that. Those were bought already. That was SFF8643 to SAS. I'm looking for SFF8643 to SATA.

You can use SFF8643 to SAS cables with SATA drives, so there is no SFF8643 to SATA with such design.

 

[Fastline]

You can use SFF8643 to SAS cables with SATA drives, so there is no SFF8643 to SATA with such design.

OMG. How could i forget this? You saved me dude. A BIG THANK YOU. You're the man!

The LSI 9400-16i uses a PCIe 3 x8 connector and has 12Gb/s

Yes, correct.

I would say that each drive has around 2 and something Gb/s, which is way more than any HDD speed.

Umm, that way it becomes 16x2Gb/s=32Gb/s and the HBA has only 12Gb/s

Anyways, here's what i understand:

I guess the normal SATA III is capable of 6Gb/s which means 600MB/s. HDDs can go up to 300MB/s (enterprise). So, for my use case, 16x300MB/s = 4800MB/s or 4.8Gb/s approx. So, as the bandwidth for the HBA is 12Gb/s, which means one can connect up to 40HDDs with almost zero impact on the HDD speed. Is that correct?

However, like you mentioned, an SSD can easily saturate this speed. Considering a normal SATA SSD which is capable of 500-600MB/s approx. If i try to install up to 20 SSDs:

20x600MB/s = 12000MB/s or 12Gb/s approx. I'm under the limit and probably no impact on either of the SSD speeds. Or is my calculation wrong here?

And if that's the case, an U.2 NVMe installed on an HBA rated for 12Gb/s speed, it can just push 1.5GB/s only? That's so much under utilized then.

Since SSDs are much faster than HDDs, they can easily saturate the available bandwidth; this can also happen with very big configurations that use cascaded HBAs.

I forgot to ask, what do you mean by cascaded HBAs? Is this something new kind of HBA or HBA configuration?

 

[Davvo]

Is the 12Gb/s bandwidth for the all SAS ports combined or is it 12Gb/s each SAS port?

I'm not sure about this, which is the reason I almost immediately edited it out in the previous post.

I forgot to ask, what do you mean by cascaded HBAs? Is this something new kind of HBA or HBA configuration?

Connect an HBA to a SAS expander.

 

[Fastline]

I'm not sure about this, which is the reason I almost immediately edited it out in the previous post.

Hmm. Seems like we need some experts here for the clarifications.

But if 6Gb/s each SATA port is the case, which i think it does or an SSD will underperform if the total ports bandwidth is 6Gb/s. This way, we know that most of the motherboards come with 6 onboard SATA ports. So, 6 portsx6Gb/s each = 36Gb/s in total bandwidth. There are a lot of motherboards since Comet Lake, where there is no bandwidth limitation between the SATA ports and/or PCIe Slots/NVMe slots. So, it means the Intel/AMD PCH is offering up to 36Gb/s (6 onboard SATA ports) for a nominal price whereas these HBA Cards with SAS ports are only capable of 12Gb/s for a huge price? Or am I missing something here?

You connect an HBA to an HBA.

OMG. How does that work though? I mean if i connect an HBA to HBA, where do i connect the drives?

 

[Davvo]

OMG. How does that work though? I mean if i connect an HBA to HBA, where do i connect the drives?

Sorry, I meant to a SAS expander. You basically gain more ports for a single port.

 

[Fastline]

Sorry, I meant to a SAS expander.

Ah ok. Got it.

Can you please check the post #55 above? Just trying to understand the bandwidth thingy. Let me know if my calculations are up to the mark.

Thanks

 

[Davvo]

Can you please check the post #55 above? Just trying to understand the bandwidth thingy. Let me know if my calculations are up to the mark.

My impression, which can be totally wrong, is the following:

• PCIe 3 x8 can transfer around 8 GB/s or close to 64 Gbps of data.
• Each port supports SAS 12 Gbps, which results in a total of 48 Gbps (close to the total amount of 64 Gbps, the difference could be the overhead)
• Each drive can get up to 12 Gbps (you can connect up to four NVMe drives with x4 lanes, one per port, or eight in x2 lanes, two per port)
• If you connect more than a single drive to a port, the connected drives split the available 12 Gbps: connecting four drives to each port results in each having up to 3 Gbps or close to 375 MB/s

But I have never used an HBA and my knowledge about them is shaky; not having much time to reasearch doesn't help.
Other users will be able to tell you more accurate info.