LSI SAS 3 (12Gb/s) issues

[wuxia]

What's the backplane firmware version?

The bug report doesn't mention. It's not submitted by me - I'm just adding it here for reference as it looks serious.

 

[Ericloewe]

Somewhat unrelated, it's hard to believe SAS3 is still in such a dubious state one year later.

 

[jgreco]

You guys, such FreeBSD noobs.

There's a REASON I'm so conservative about moving forward with upgrades, etc.

 

[wuxia]

Coming from Linux I admit all those hardware idiosyncrasies are a little bit unexpected :)

 

[jgreco]

Linux isn't that much better. There's plenty of crap-grade drivers for crap-grade hardware there too. That's the problem with the PC architecture. There's no motivation for vendors to produce good silicon, good documentation, or (shudder) high quality device drivers, unless you're someone like Intel where your goal is to dominate.

 

[tomarch]

Hi, I'm the guy which report the issue. I'm still looking for a solution and i can give you any usefull information or making any test.
How i can retrieve the backblane firmware ?

 

[Ericloewe]

Hi, I'm the guy which report the issue. I'm still looking for a solution and i can give you any usefull information or making any test.
How i can retrieve the backblane firmware ?

Generally, the manufacturer has a tool to talk to the backplane and do stuff like update the firmware.

 

[Ceetan]

So what's the current state of the LSI 3008 controller?

Would be useful to know as I am

currently deciding between

Supermicro X11SSL-CF

and

Supermicro X11SSH-F


For my (potentially) eight Drive build.

(in the case of the former I would use the controller in IT mode and a solid state drive I already have laying around as a boot device, whereas the latter would require that I purchased and M.2 Drive)

 

[TXAG26]

I've had two in service for almost two weeks now and ran a ton of scrubs on two different 8-disk RaidZ2 pools (pools were around 75% filled up). I was unable to trigger any type of ZFS error. Both servers had the HDDs direct-attached. Conservatively, with all of the scrubs, we probably passes about 400TB of data through the two LSI 3008. No issues. Not scientific by any means, but it gave me confidence in the hardware. Make sure to grab the latest v12.00.02.00 IT firmware from Supermicro's ftp site. [fixed incorrect firmware version reference].

ftp://ftp.supermicro.com/Driver/SAS/LSI/3008/Firmware/

 

[depasseg]

Make sure to grab the latest v20 IT firmware from Supermicro's ftp site.

Supermicro is up to v20 for the 3008? That sounds like a 2008.

My experience with a 3008 has been fine for over a year (but only connected to a SAS-3 backplane).

 

[TXAG26]

Supermicro is up to v20 for the 3008? That sounds like a 2008.

My experience with a 3008 has been fine for over a year (but only connected to a SAS-3 backplane).

I'm sorry for the confusion, I got my controllers mixed up. Yes, the 2308 is on PH20. The 3008 is on PH12, which corresponds to driver v13 in FreeNAS. PH12 IT is what you need on the 3008 controller for FreeNAS v9.10.xx.

ftp://ftp.supermicro.com/Driver/SAS/LSI/3008/Firmware/

 

[Ceetan]

I recently got this from my server. Can't be good, can it?
possible LSI3008 Issue?

 

[Ericloewe]

I recently got this from my server. Can't be good, can it?
possible LSI3008 Issue?

You'll have to provide more detail. How the drives are connected, Firmware versions, etc.

Ideally, system specs, too, since signatures aren't shown on mobile.

 

[Ceetan]

You'll have to provide more detail. How the drives are connected, Firmware versions, etc.

Ideally, system specs, too, since signatures aren't shown on mobile.

Right


FreeNAS 9.10-u2 Stable
MOBO: Supermicro X11SSL-CF with SAS3008 flashed to IT mode with p12 firmware. All drives connected to the SAS-controller via supermicro CBL-SAST-0556 breakout cables
CPU: Intel Core i3-6100
RAM 2*16GB Crucial DDR4 ECC 2133MHz (CT2K16G4WFD8213)
8* WD Red WD40EFRX 64MB 4TB in RAIDZ2
BOOT 90GB Corsair SSD
CASE: Fractal design node 804
PSU:EVGA SuperNOVA 650 G2

 

[Ericloewe]

How long are the cables?

 

[Ceetan]

Hmmm....that could be the issue actually. 4 of the 8 are 90 cm. Wierd tha it only seems to affect 3 of them, in that case though.

 

[Ericloewe]

Yeah, that's marginal. Try to keep it under 50cm, if possible.

 

[Ceetan]

Yeah, that's marginal. Try to keep it under 50cm, if possible.

I just contacted the retailerer to see if I can exchange my cables for a shrter model. It is a bit perpelxing though that it seems to affect only 2 drives, when 4 are conected to the 90 cm cables. Could that simply be a mixture of bad luck and the mragnal length?

 

[Ericloewe]

I just contacted the retailerer to see if I can exchange my cables for a shrter model. It is a bit perpelxing though that it seems to affect only 2 drives, when 4 are conected to the 90 cm cables. Could that simply be a mixture of bad luck and the mragnal length?

Yeah, the SATA spec is very tight on the cables. It's part of why there isn't a SATA 12Gb/s, it'd be just too difficult without changing everything.

 

[TXAG26]

I too would lean towards bad cables. I've had a couple rounds of dodgy cables over the years, even when buying good quality Adaptec/LSI/Amphenol type cables. It's just the nature of the beast. I usually try to use the shortest cable that will do the job (within reason).