mrsas driver instead of mfi driver

[drstreet]

You want neither of those. You want mpr.

Crossflash your card to IT firmware.

...but I can't yet understand why on FreeBSD 9.3 mrsas(4) man page is explicitly written:

The mrsas driver supports the following hardware:

[ Thunderbolt 6Gb/s MR controller ]
...

[ Invader/Fury 12Gb/s MR controller ]
+o LSI MegaRAID SAS 9380
+o LSI MegaRAID SAS 9361
+o LSI MegaRAID SAS 9341
+o DELL PERC H830
+o DELL PERC H730/P
+o DELL PERC H330

 

[jgreco]

because there is a difference between "supports" and actually being the preferred driver.

 

[drstreet]

ok jgreco, but I can't understand exactly what is this difference. Could you explain, please?

Thanks

 

[jgreco]

My recollection is that mrsas(4) is LSI's new attempt at a unified driver for their current and recent product lines. The old mfi(4) driver supports some of those cards too.

This does not imply that mrsas is a better choice, just a newer driver.

Also, the H330 is, I think, actually an LSI 3008 controller. As with cards like the M1015, this ideally needs to be bopped into being a pure HBA, which might actually cause it to be covered by a third driver, which is escaping me at the moment, EINSUFFICIENTCOFFEE.

It's not horribly unusual for two drivers to support a given bit of hardware. We've seen that in the past with de(4)/dc(4), ncr(4)/sym(4), etc.

 

[Ericloewe]

My recollection is that mrsas(4) is LSI's new attempt at a unified driver for their current and recent product lines. The old mfi(4) driver supports some of those cards too.

This does not imply that mrsas is a better choice, just a newer driver.

Also, the H330 is, I think, actually an LSI 3008 controller. As with cards like the M1015, this ideally needs to be bopped into being a pure HBA, which might actually cause it to be covered by a third driver, which is escaping me at the moment, EINSUFFICIENTCOFFEE.

It's not horribly unusual for two drivers to support a given bit of hardware. We've seen that in the past with de(4)/dc(4), ncr(4)/sym(4), etc.

SAS3 IT/IR mode is mpr(4)

 

[jgreco]

yeah, that's right, you keep going on the alphabet... how could I have forgotten that

 

[mjws00]

I'd say go slow with that H330. We've seen no crossflashed Dell 3008's. Only IT mode on SM -CLN4's and a couple actual LSI cards. Your mezz slot may check for Dell firmware depending on the server. Even if you don't brick it... You still end up on a pre-alpha driver.

I'd drop the card until there is proper support. There is literally no upside to the risk, and any proven LSI HBA is a much better solution. Dell will sell you a 9207-8i if you are locked in.

 

[Ericloewe]

I'd say go slow with that H330. We've seen no crossflashed Dell 3008's. Only IT mode on SM -CLN4's and a couple actual LSI cards. Your mezz slot may check for Dell firmware depending on the server. Even if you don't brick it... You still end up on a pre-alpha driver.

I'd drop the card until there is proper support. There is literally no upside to the risk, and any proven LSI HBA is a much better solution. Dell will sell you a 9207-8i if you are locked in.

The funny part is that the H330 stock firmware seems to allow JBOD operation, judging by another recent thread. It was never fully confirmed, but it's an interesting option to verify.

 

[mjws00]

The Dell firmware supposedly passes through nicely. But what glitches will a firmware/driver mismatch introduce? That chipset doesn't always play nice even when the right things are done. I'm always happy to have a guinea pig ;) I'd be burning in hard core and no where near data.

 

[viniciusferrao]

The Dell firmware supposedly passes through nicely. But what glitches will a firmware/driver mismatch introduce? That chipset doesn't always play nice even when the right things are done. I'm always happy to have a guinea pig ;) I'd be burning in hard core and no where near data.

I'm with a H330 and I can't passthrough it correctly and I can't even flash the card... The loaded driver on the machine is the mfi one.

 

[jgreco]

Yeah, that's no good.

 

[viniciusferrao]

Yeah, that's no good.

I think we've done.

Take a look at this bug report: https://bugs.freenas.org/issues/11764

I've noted that the stable release already fixed this bug. So I gived a try and enabled the mrsas support on tunables of FreeNAS webinterface:
hw.mfi.mrsas_enable = 1
mrsas_load = YES

After the update to latest 9.3-STABLE release and setting up the tunables the mrsas(4) driver was loaded successfully and the warning about `smartd` not running was lifted.

I'm running an stress test with iozone to try to reproduce the bug #11764. But at least for me appears to be fixed, and of course after enabling the mrsas(4) the disks appeared like /dev/daX and smart was working. I even started an SMART long test on each disk and everything appears to be fine!

Let-me do some additional tests and see how it behaves.

 

[jgreco]

The problem is that even when the mrsas driver was "supposed" to work (back in FreeNAS 8 days), it didn't really work reliably in various scenarios. I think the thing that is most dangerous would be what happens in a situation where the driver splorfs and suddenly part of your pool goes offline. I know I don't care to be the guinea pig for this!

 

[viniciusferrao]

The problem is that even when the mrsas driver was "supposed" to work (back in FreeNAS 8 days), it didn't really work reliably in various scenarios. I think the thing that is most dangerous would be what happens in a situation where the driver splorfs and suddenly part of your pool goes offline. I know I don't care to be the guinea pig for this! :)

Yep this is f***ing bad. The issue you described on 8.0, are you able to reproduce the issue on the past? If yes I would like to try on this pool to check if the behaviour still happens.

And just for the sake of completeness: there are some bug reports of the mrsas(4) dropping disks after some commands, like an SMART check. I've checked all bugs submitted to FreeNAS bugs with mrsas on it and tried to reproduce them, nothing wrong at this moment. But definitely I'm not comfortable running the pool this way.

 

[jgreco]

I don't run any FreeNAS boxes with LSI cards in mrsas mode, so I have no idea as to which misbehaviours might still happen. From my perspective, if I have a pool of valuable data, I am not interested in being a guinea pig for driver reliability testing, and since we heavily disrecommend those cards in mrsas mode, very few other people choose to either. The truth of it is that it's quite possible for a card to appear to be problem-free for tens of thousands of hours before something breaks in just the right way to tweak an edge case. We know, however, that the IT mode stuff works well because there are cumulatively over a hundred million hours of operational time on these cards that haven't yielded adverse results.