Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

[jgreco]

I got a LSI 9217 card and without even any drives plugged in, the heatsink is so hot I almost burned my finger.
Is this normal? Are the chips supposed to be this hot?

The LSI cards are typically intended to be run in a server with substantial forced airflow. If you do not have airflow past them, they typically go thermonuclear. The typical LSI chipset is dissipating something around 10 watts, so it does not really take a huge amount of airflow to cool them, but if you create a zone without airflow, they do suffer and can even do bad things.

Many, but not all, LSI cards come with vented slot covers which hopefully your card has. If you are not building inside a regular server chassis, you may need to arrange cards to create airflow. Some people advise to add a chipset fan to the LSI card, but in general I think this is a bad idea, as the fan will eventually cook, and then the LSI will cook worse, and this can potentially spew spurious write errors into your pool.

I keep intending to order some custom double-height LSI heatsinks but there's some variety in the shapes and the cost is nearly unjustifiable. Plus I generally deploy these things in real server chassis.

 

[jgreco]

Thank you very much, @jgreco . I've been playing with FreeNAS for a year, now, have read this page twice before, and it was lost on me. I just ordered a new Super Chassis with a TQ backplane, and this helped clarify a lot of pieces that have been floating around. Looks like I'll need an expander to round out my system, and now I know.

You are entirely welcome to tell me where I failed to make things clear. I try my hardest to place myself in a "newbie's" shoes when writing this sort of thing. That does not mean I'll be successful. But you should definitely assume it is my intent to make this clear to anyone who is in your situation, and telling me where I failed is beneficial to everyone.

 

[Octopuss]

The LSI cards are typically intended to be run in a server with substantial forced airflow. If you do not have airflow past them, they typically go thermonuclear. The typical LSI chipset is dissipating something around 10 watts, so it does not really take a huge amount of airflow to cool them, but if you create a zone without airflow, they do suffer and can even do bad things.

Many, but not all, LSI cards come with vented slot covers which hopefully your card has. If you are not building inside a regular server chassis, you may need to arrange cards to create airflow. Some people advise to add a chipset fan to the LSI card, but in general I think this is a bad idea, as the fan will eventually cook, and then the LSI will cook worse, and this can potentially spew spurious write errors into your pool.

I keep intending to order some custom double-height LSI heatsinks but there's some variety in the shapes and the cost is nearly unjustifiable. Plus I generally deploy these things in real server chassis.

Hm.
That's going to be interesting.

There's not really any airflow at the top of the case so I'll have to observe once the server is fully up and running (don't have any disks ready for the NAS yet).
The case (Phenom M) is a little obscure when it comes to airflow. And I don't really intend to add anything to the top side (not if I can help it).

Maybe the heatsink is doing a good job though, because while I almost burned my finger touching it, the underside of the card is just somewhat warm.

 

[danb35]

Looks like it might be worth coming up with a way to mount a fan to that heatsink.

 

[Stux]

There’s a fan mount at the top of your picture. Put a fan there and have it blow at the lsi Heatsink. Probably doesn’t have to be spinning fast.

 

[Octopuss]

Do these cards have any temperature sensors or something by chance?

 

[jgreco]

Do these cards have any temperature sensors or something by chance?

Depends on the generation of the card. I think they started doing that with the SAS3 HBA's. Some of the SAS2 RAID controllers (926x/927x/928x) do, some don't. Most of the HBA's that are good for FreeNAS are what LSI considered "entry-level" cards so they didn't put fancy features on them.

 

[ere109]

You are entirely welcome to tell me where I failed to make things clear. I try my hardest to place myself in a "newbie's" shoes when writing this sort of thing. That does not mean I'll be successful. But you should definitely assume it is my intent to make this clear to anyone who is in your situation, and telling me where I failed is beneficial to everyone.

It isn't so much a failure on your part. These are complex concepts, and until one ever enters the arena and starts exploring to understand the different hardware options, they're not ready for this.
What helped me was the decision to move to hot swap. Suddenly i was reading about LOTS of equipment and technology. So i now knew the terms, but was unsure what they did. Reading your post filled in those blanks.
I ordered a device with a TQ backplane, and suddenly understood. My HBA has 8 sas ports, but I saw the SFF connectors on certain hardware, so then understood the difference between interfaces, and read another post about forward and backward cabling (can't recall if you address that). The only possible suggestion I might have would be a simple bullet point index at the front to break down the chronology of SAS design, and perhaps a note on expander vs second HBA (cost vs capacity). When a reader is ready for it, this is a great place to turn for answers.

 

[subhuman]

A breakout cable allows this. This is a SFF8087-to-single-SAS breakout cable; this is a SFF8643-to-single-SAS breakout cable.

Are you sure of this? Those look suspiciously like SATA connectors to me.

 

[Chris Moore]

Are you sure of this? Those look suspiciously like SATA connectors to me.

The pictured cables have four SATA connectors on the drive end and SAS connectors on the controller end.

 

[subhuman]

The pictured cables have four SATA connectors on the drive end and SAS connectors on the controller end.

Right, wouldn't that mean they're SFF8087/SFF8643-to-single-SATA cables then, not SFF8xxx-to-single-SAS as currently stated in the first post?

 

[Chris Moore]

Right, wouldn't that mean they're SFF8087/SFF8643-to-single-SATA cables then, not SFF8xxx-to-single-SAS as currently stated in the first post?

I am sure that @jgreco will consider this when he has a bit of spare time.
The SFF8087 references the type of connector on the controller and it is clear to me what is meant, but I can see how it could be confusing.
Here is a link to an example product, for the photos, not that I am suggesting this vendor:
Cable-Matters-Internal-SFF-8087-Breakout
Where, if it were SFF8087 to SAS, it would be this product, again for the photos, not that I am suggesting this vendor:
Connector-Creation-Internal-SFF-8087-connectors

 

[blueether]

I am sure that @jgreco will consider this when he has a bit of spare time.
The SFF8087 references the type of connector on the controller and it is clear to me what is meant, but I can see how it could be confusing.
Here is a link to an example product:

Robot Check

Where, if it were SFF8087 to SAS, it would be this product instead:

Amazon.com: CableCreation Mini SAS Cable with SATA Power, Internal Mini SAS 36 Pin to 4 x 29 Pin Cord with SATA Power, SFF-8087 to SFF-8482 Cable, 3.3 FT: Computers & Accessories

Amazon.com: CableCreation Mini SAS Cable with SATA Power, Internal Mini SAS 36 Pin to 4 x 29 Pin Cord with SATA Power, SFF-8087 to SFF-8482 Cable, 3.3 FT: Computers & Accessories

www.amazon.com

I think that the confusion comes from the fact that the SATA end will still do SAS if connected to a backplane that has SATA style conectors and not 8087 mini-sas, I think...

 

[jgreco]

Are you sure of this? Those look suspiciously like SATA connectors to me.

10Mbps, 100Mbps, 1Gbps, 10Gbps ethernet, RJ45 (USOC), RJ48 (TIA-968-A) T1, ATM, ADSL, ISDN, Serial (TIA-561), and many other things all appear on an 8P8C modular connector, but they are *ALL* extremely different things, and none of them are compatible with each other (with the possible exception of ethernet where sometimes hardware supports several speeds). Appearances are deceptive.

SFF-8643 was originally aimed at 12Gbps SAS but has been seen used for 6Gbps SAS, NVMe, and 6Gbps SATA.

You can look at any number of mainboards and find discrete single-lane SAS connectors. Hint: they *look* suspiciously like SATA connectors, and because SAS supports SATP, your SATA devices will work when hooked up to them. But -- shockingly -- so will SAS devices. SAS devices will not work on a SATA port but SATA devices will work on a SAS port. SAS is effectively a superset, so, any time SAS works, it is more correct to say that it is SAS.

So, if your complaint is that you cannot plug the shown 7 pin SAS breakout directly into a SAS drive, you're correct, because a SAS drive typically has a secondary connector (pictured previously in this thread) which would prevent the insertion of the single-lane SAS cable. However, the single-lane SAS breakout plugs into SAS backplanes (such as the ever popular -TQ's) just fine, and a SAS drive will happily plug into that, so these clearly have extensive use cases, even if the applications for the breakout cables are not universal.

Additionally, in a crisis, you can peel away the plastic edge of the 7-pin "SATA" connector and plug it directly into a SAS drive, because it is carrying SAS signalling. Of course that only works if it is carrying SAS signalling.

 

[jgreco]

I think that the confusion comes from the fact that the SATA end will still do SAS if connected to a backplane that has SATA style conectors and not 8087 mini-sas, I think...

I'd say that the confusion comes from the fact that someone is confusing connectors and signalling.

Back in the early days of SATA and we were building the first 24-drive-in-4U SATA servers, we were using the 3Ware 9550SX-12ML cards, two of them, to drive the 24 bays. These used SFF8470 cables and the "SATA-blessed" versions were unobtainium, so we used SFF8470 Infiniband cables instead (and it was just fine). The thing is, you have to realize that an SFF8470-to-8470 cable should be roughly the same whether it is running Infiniband, SAS, SATA, etc.

 

[blueether]

So, if your complaint is that you cannot plug the shown 7 pin SAS breakout directly into a SAS drive, you're correct, because a SAS drive typically has a secondary connector (pictured previously in this thread) which would prevent the insertion of the single-lane SAS cable. However, the single-lane SAS breakout plugs into SAS backplanes (such as the ever popular -TQ's) just fine, and a SAS drive will happily plug into that, so these clearly have extensive use cases, even if the applications for the breakout cables are not universal.

That is what I was trying to say, but as always you explain things with very good examples

 

[subhuman]

Additionally, in a crisis, you can peel away the plastic edge of the 7-pin "SATA" connector and plug it directly into a SAS drive, because it is carrying SAS signalling. Of course that only works if it is carrying SAS signalling.

So... because you can either install it with a hammer, or cut it to make it fit somewhere it's not intended to be, you consider it to be a SAS connector?

Let me quote your entire paragraph that my earlier excerpt came from:

A multilane connector may be broken into its four individual lanes using a breakout cable. For example, if you get an SAS HBA, it probably comes with one or two SFF8087's on it, but you may want to directly attach hard drives. A breakout cable allows this. This is a SFF8087-to-single-SAS breakout cable; this is a SFF8643-to-single-SAS breakout cable.

(bold added for emphasis)
You're talking about connecting that end to hard drives. That connector is not intended to connect to SAS hard drives, you just said it above- you have to modify the connector to force it into something it's not intended for. It is, however, intended to connect to SATA drives. It is, as pictured, not cut apart and modified, intended to connect to SATA drives. Not SAS.

Do you seriously do this? I don't even want to know what you do for power delivery if you do.

 

[Ericloewe]

Why is this a point of contention? If you want to connect directly to SAS disks, get an SFF-8643/SFF-8087 to SAS-style connector (the one with power attached) breakout cable. If you want to connect to SATA devices or backplane or whatever, get one with SATA-style connectors.

 

[subhuman]

Why is this a point of contention? If you want to connect directly to SAS disks, get an SFF-8643/SFF-8087 to SAS-style connector (the one with power attached) breakout cable. If you want to connect to SATA devices or backplane or whatever, get one with SATA-style connectors.

I agree with what you're saying about breakout cables. The issue is that in the first post in this thread, it states they can be used to connect directly to drives. What is pictured is the latter (SATA connectors) but the text calls it the former (SAS).
For anyone who has worked with either even in passing, it's no big deal. But in a topic that is calling itself a primer for people not familiar with SAS, it can be a problem. A new person, following what's currently posted, buying those cables and trying to connect them to SAS drives- which is what the text tells them they're for- is going to break hardware (if they force it) and/or have a non-functional setup (from a half-plugged-in connector).
Why is it a point of contention? I don't know why it should be. It seemed like a minor mistake, either in image choice or in the descriptions.

 

[Ericloewe]

Ah, I see what you mean. I do agree that it is a bit confusing if you only read that part. I think it's okay if you read the whole post, but I'm biased by the fact that I know this stuff.