Broadcom 9600W-16e not supported (yet)?

[jgreco]

Broadcom's support isn't really great for the time being, they claim they have the driver but i nowhere see the debian driver.

That's fine. TrueNAS is an appliance and you cannot install your driver of choice. You get the one that comes with TrueNAS and you should also make sure you're using the correct firmware on the card. See the forum Resources section for the latest v16.00.12.00 for the 12Gbps cards.

On top of that their support started talking about freebsd while i specifically mentioned it was truenas scale running on debian ...

I would imagine that they get a relatively high quantity of TrueNAS related questions. Sometimes people don't read to the end or even understand that there are both FreeBSD and Debian versions of TrueNAS. They are probably used to handling these a certain way.

 

[zormik]

That's fine. TrueNAS is an appliance and you cannot install your driver of choice. You get the one that comes with TrueNAS and you should also make sure you're using the correct firmware on the card. See the forum Resources section for the latest v16.00.12.00 for the 12Gbps cards.



I would imagine that they get a relatively high quantity of TrueNAS related questions. Sometimes people don't read to the end or even understand that there are both FreeBSD and Debian versions of TrueNAS. They are probably used to handling these a certain way.

Thanks for the hint, i'll be receiving the 9400 somewhere in the middle of next week so I'll make sure to take a look :)

 

[Ericloewe]

On top of that their support started talking about freebsd

What did they say about FreeBSD support for these things?

 

[zormik]

What did they say about FreeBSD support for these things?

this was the reply:

The 96xx series have a unified driver which can be used by both, MegaRAID and HBA controllers.
For Debian you can use the HBA driver sources available on our web sites, you need to compile the source codes.
For FreeBSD , you can use the drivers from 9670w RAID Controller. Link is attached.

https://www.broadcom.com/products/storage/host-bus-adapters/sas-nvme-9600w-16e
https://www.broadcom.com/products/storage/raid-controllers/megaraid-9670w-16i

You also need to verify that the freeBSD kernel version of TrueNAS is the same as in the driver package.

 

[jgreco]

I don't know what the attitude at iX has been about this recently; new device drivers for storage are a rare thing. I suggest putting this into a bug report and seeing what happens.

 

[Ericloewe]

I checked and the driver doesn't seem to be upstreamed into FreeBSD yet. But since Broadcom is kind enough to provide a driver, it does sound like filing a ticket might be interesting.

That said, we still haven't heard about whether the driver is in TrueNAS Scale yet...

 

[jgreco]

That said, we still haven't heard about whether the driver is in TrueNAS Scale yet...

Not clear why anyone would care. It would probably be prudent to use the updated driver they provided.

 

[zormik]

I posted a support ticket with IX for the 9600 series just for the sake of it. I'll be using the 9400-8e next week because my old server is close to exploding

 

[alex992]

is there need flash fimware with specific version # for 9400-8e for Core?

 

[Ericloewe]

Whatever the latest is for the 94xx cards.

 

[jgreco]

Whatever the latest is for the 94xx cards.

Are we sure about that? I would think 16.00.12.00 would still be the preferred.

 

[Ericloewe]

It's hard to say, they just gave up on any sort of useful communication on this topic before those cards showed up in peoples' hands. I do think the last statement was something to the effect of "just use the latest", which is 16.00.12.00 for 93xx. I think 94xx has newer firmware and know that 95xx do have newer firmware.

 

[jgreco]

I think 94xx has newer firmware and know that 95xx do have newer firmware.

Gives me an uneasy feeling.

 

[Ericloewe]

@morganL, you guys are using 94xx and 95xx cards - what's the current party line on firmware versions?

 

[alex992]

how to flash 94xx or 95xx for new firmware? is same EFI sas2flash as for 92xx will work?

 

[Ericloewe]

No, but sas3flash should work.

 

[alex992]

With same steps as sas2flash ?

 

[Ericloewe]

See https://www.truenas.com/community/r...9300-9305-9311-9400-94xx-hba-and-variants.54/

 

[zormik]

Short update, i replaced the 9600w-16e with the 9400-8e and it works perfectly now.
For those who are having a 9600 series, this is the feedback from iX:

Hello,
We cannot update the kernel version for out stable version (Bluefin).
A new version will be used for Cobia and I hope your card will then be supported.

Thank you everybody for helping me out, and hopefully this thread can help others with the same issue in the future :)
Z

 

[zormik]

Everything is working well bar one thing, i noticed the performance is really low on truenas scale, my old setup (truenas core) which is almost 8 years old is much faster then my new one. I'm using 5 x WD DC HC 570 22TB in a raidz1 with write performance of around 250MB/s and read performance of around 140MB/s.

If i do a second read of the same 35GB large file the ram kicks in and my nic's get maxed easily (5gbit NIC on client side). So it really seems like it's HDD/SAS related. Any ideas that could help me figure this one out?

Thx,
Z