TrueNAS Core With - hp smart array p411 controller

[LHBL2003]

Hello,

I just installed my first TrueNAS Core.

I am using the following hardware for the test:

DL380 G7 with vShpere 7.0.3
HP Smart Array P411 Controller (PCI)
HP StorageWorks D260 Array 12 Bye AJ940-63002 (12 HDD Storage)

I have set up for the HP Smart Array P411 Controllers in ESXi under Host -> Manage --> Hardware --> PCI Devices --> HPE Smart Array P411 --> Passthrough --> Active
Enabled

When booting from Truenas, Smart Array P411 is listed.

Only I can't find my 12 HDDs. No matter if I create a Rain5 in the Raid Controller or configure no Raid.

Is it possible to add such a raid controller and storage to TrueNas? And where can I find an entry point for the configuration?

Under Storage and Pools I find nothing?

Thanks for the help

 

[Ericloewe]

If you had done a little bit of reading, you'd know that RAID controllers are a no-go. Please read the linked documentation for details on what you should be doing and come back here if you have any questions.

HP StorageWorks D260 Array

It's not clear to me that your disk chassis is suitable. If it's just an expander chassis, with all RAID logic on the host side, and if the expander actually works correctly, it should work - in the same vein as the fairly popular Dell MD1200.
However, if it's something "fancier" it's likely to be problematic.

 

[Borja Marcos]

It should be handled by the ciss(4) driver I think.

At your own peril, and making sure that you do not try to define any logical volumes or stuff, (repeat that phrase 1000 times, rinse and soak)
maybe you can try this tunable setting:

hw.ciss.expose_hidden_physical=1

I have been doing this for many years with different RAID cards and it worked flawlessly. But, again, don't ever let the RAID part of the card touch your disks.

 

[Ericloewe]

The ciss stack just isn't good enough. Unless there's been a massive improvement, which nobody has claimed - much less proven - it's just not an adequate choice.

 

[Borja Marcos]

The ciss stack just isn't good enough. Unless there's been a massive improvement, which nobody has claimed - much less proven - it's just not an adequate choice.

Hmm I see. I have considered the "it´s the only thing I´ve got" factor when recommending the somewhat risky device passthrough option. So do you think it´s more dangerous than doing it with, say, mfi or aac?

 

[Ericloewe]

I make no particular claims about relative danger, other than to put everything other than AHCI (with reputable hardware in tow) and mpr/mps in the "sucky at best" pile, with maybe a carve-out for mrsas, again depending on the specific hardware.

Done properly, I suspect it'd be difficult to kill a pool with any of these stacks. But the fact that it's not mpr/mps shows already some cut corners, so that doesn't bode well for "done properly".

That said, I 100% welcome anyone who just wants to play around and test things, to see if they've improved. But it's important to separate that from production use cases.

 

[Borja Marcos]

Understood :) I was asking because I ran a pool on an aacraid card using passthrough (by manually patching the driver to expose the disks to CAM) for 10+ years without a single flaw.

It had quite a heavy load, performance was good and it never complained at all.

ZFS has proven to be insanely resilient. We also suffered a HBA failure (it was a mps). Sometimes, with heavy load, it would corrupt data. Yet ZFS corrected the checksum errors and no single bit was damaged!

 

[Elliot Dierksen]

True, but the replacement of the RAID card with a true HBA is probably a $50ish item on eBay. If it were more expensive, that would be a different point.

 

[LHBL2003]

So first of all thanks to @Ericloewe.
If you don't know what to look for with TrueNas, you first look for things that have to do with your hardware.

Apparently no one has tried it with my equipment or not documented online.
I would still try it out, as long as it runs stable, as this would be an inexpensive grab for non-relevant data. The hardware is lying around and no longer needed. Alternatively I could use a Windows Server operating system, but then you don't get to know TrueNAS :)

I have a couple of approaches that I will test the days.

Supposedly there should be an update for the P411 card to officially activate HBA. As I understand it, this is exactly what TrueNAS needs or not?

https://support.hpe.com/connect/s/softwaredetails?language=de&softwareId=ux_116282_2

HBA/RAID mode toggle support is now available within ORCA. (Mode Toggle only supported for P411 and P410i controllers).

This article describes the procedure to enable the HBA mode. However for the RAID card P420i / H240. These are used for the front HDD boxes of the DL380 server. But maybe the procedure for my P411 is identical.
https://forums.unraid.net/topic/91922-convert-hp-smart-array-controller-to-hba-mode/

I will report when I have found out more.

 

[LHBL2003]

@Borja Marcos

You mean you should specify this in /boot/loader.conf?
hw.ciss.expose_hidden_physical=1


What does it do?
What does this do / What do I see afterwards?

THX

 

[LHBL2003]

PS: According to the documentation, the P411 is supported by Ciss(4).

ciss(4)

www.freebsd.org

 

[Elliot Dierksen]

PS: According to the documentation, the P411 is supported by Ciss(4).

Yes, but the point that is being stressed is that the quality of the ciss driver in FreeBSD is not good. Even good hardware performs poorly if the driver for the OS in question is poor.

 

[LHBL2003]

Currently I can't manage to set the P411 to HBA mode.
Is such an LSI card theoretically compatible with TrueNAS?
I need a SFF-8088 connection.

SUPERMICRO SAS9200-8E - LSI SC08E 6GB 2-Port SAS EXT HBA​

SUPERMICRO SAS9200-8E - LSI SC08E 6GB 2-Port SAS EXT HBA

LSI SC08E 6GB 2-Port SAS EXT HBA

www.kimbrer.de

 

[Ericloewe]

Sure, that would work.

 

[Borja Marcos]

You mean you should specify this in /boot/loader.conf?
hw.ciss.expose_hidden_physical=1


What does it do?
What does this do / What do I see afterwards?

Sorry about the belated answer. I missed it.

RAID cards usually work as HBAs as well. In order to avoid potentially "lethal" confusions, they don't expose disks as HBA reachable devices. If you enable that flag you tell the driver to expose them. The risk is, if you also configure some RAID stuff there might be two entirely different software instances accessing the same disks (the RAID card firmware and the operating system disk driver). You can imagine the consequences.

Also, there is a second potential problem if the card firmware somehow "filters" SAS messages due to some odd design decision.

I have done that safely with LSI and formerly Adaptec cards but I have no idea whether it can be realistically safe with CISS. And after the remarks by @Elliot Dierksen I would be extremely skeptic. So, better to find a replacement card.