Raid controller for Scale

[hyi]

I plan to install TrueNas Scale on a HPE ML110. The computer comes with P440 raid controller. Understood that LSI card is the most recommended one for FreeNas core, which is based off FreeBSD. Since Truenas scale is based off Linux, and HPE supports P440 in Linux, can I just use the controller in HBA mode instead of purchasing a LSI raid card? Thanks,

 

[jgreco]

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

 

[hyi]

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

Thanks for the link to the post. I have read it before, based upon the content and the date of the post, my impression was that it was mainly for FreeBSD. I thought one of reasons that TrueNas has been ported to linux, besides VM and docker supports, is that more hardware drivers in Linux. HP P440 could be a good example here because the manufacturer does support it in Linux. Besides the card doesn't have enough history in TrueNas Scale, is there any showstopper?

 

[skittlebrau]

If it can operate in purely IT/HBA mode and the disks and their relevant SMART data show up individually in TrueNAS SCALE, then it should be fine.

 

[sretalla]

If it can operate in purely IT/HBA mode

Did you read the post from @jgreco?

If you meant "can be flashed into IT mode", fair enough.

 

[skittlebrau]

Did you read the post from @jgreco?

If you meant "can be flashed into IT mode", fair enough.

Yep that was my intention.

However I would also add that for the sake of saving potential headaches, I would always use whatever is tested and proven wherever possible.

 

[Ericloewe]

Thanks for the link to the post. I have read it before, based upon the content and the date of the post, my impression was that it was mainly for FreeBSD. I thought one of reasons that TrueNas has been ported to linux, besides VM and docker supports, is that more hardware drivers in Linux. HP P440 could be a good example here because the manufacturer does support it in Linux. Besides the card doesn't have enough history in TrueNas Scale, is there any showstopper?

Linux helps to some extent. It's conceivable that some other HBA series would work. But then we're back to "why not LSI and be done with it?", and it's difficult to justify anything else. Sure, if someone had a 16-port card at 30 bucks, that might be interesting. Or maybe much lower power consumption or something. But nobody really does, so LSI still rules supreme.
In addition, all the comments regarding RAID still apply. RAID cards' HBA modes suck as much under Linux as they do under FreeBSD (that's not to say that there isn't a model out there that actually works - but again, why pay more for no benefit?).

 

[jgreco]

HBA modes suck as much under Linux as they do under FreeBSD (that's not to say that there isn't a model out there that actually works - but again, why pay more for no benefit?).

There's a reason why ZFS-based projects generally warn people off RAID cards. Proxmox does, TrueNAS does, etc. Basically it isn't good enough that a RAID controller sees the disks and presents them to the OS. It has to do it VERY WELL. And if it does it VERY WELL, you're golden.

But evaluating this is something far beyond the abilities of a typical SOHO user, and you should be noticing that even those of us with access to LOTS of various hardware have a very short list of acceptable solutions, which basically includes AHCI controllers (and preferably Intel ones at that), LSI 6Gbps/12Gbps HBA with modern disks (because even the HBA's have some weird incompatibilities), and Intel PCH SCU, which only comes as a built-in option on mainboards, so is available if you happen to have one of those boards, but otherwise isn't a viable solution.

based upon the content and the date of the post, my impression was that it was mainly for FreeBSD.

Yes, it's "mainly for FreeBSD", because at the time the post was written, FreeBSD was all there was.

HP P440 could be a good example here because the manufacturer does support it in Linux.

It's supported in FreeBSD too. Do you have a point?

Besides the card doesn't have enough history in TrueNas Scale, is there any showstopper?

Oh. Dear. Lord.

<super-frustrated> No, go ahead, just use it. Of course, the hardware and firmware are magically better and magically do not have any of their inherent problems endemic to RAID controllers when used under Linux, because Linux is magic and FreeBSD only has problems because FreeBSD sucks. Your hardware, when plugged into a magic Linux system, will have magic flow from the Linux kernel, infusing the crappy RAID card firmware and design with unspecified magic Linux influences that will make it work fine under magic Linux and even though it will only have a few thousand hours under its belt when a disk fails or some other adverse event happens, and you find out firsthand the hard way whether or not that particular failure kills you. Because yes there's a reasonable chance it COULD be fine, but there's also a chance it won't be. We need brave volunteers to put their data on the line and risk their bits for the good of the community.

Oh dear lord I suppose I need to point out I'm being exasperatedly sarcastic. DO NOT DO THIS. I AM NOT ACTUALLY TELLING YOU TO DO THIS. IF YOU VALUE YOUR BITS, DO NOT DO THIS.

Please, someone explain to me, why it is people are so determined to find reasons to disregard the things I've taken the time to author in an effort to save them from their dangerous impulses.

 

[Etorix]

Please, someone explain to me, why it is people are so determined to find reasons to disregard the things I've taken the time to author in an effort to save them from their dangerous impulses.

I'm afraid you already explained it above: It's the magic flow from "Linux". People come in because they've heard that TrueNAS SCALE is Linux, so it must be good and disregard everything that was written because it was for CORE and FreeBSD. They don't even need to be Linux fanboys, it suffice they only know of Linux and, perhaps, Windows because ignorance is bless; they don't know about BSD, or any other proper Unix for that matter, because the Linux way is supreme, and what they don't know, they disregard. Never mind that Internet infrastructure ran on BSD long before these newcomers were born… BSD is "obscure stuff" because it's not Linux. Full stop.

And it's just the beginning…

 

[Arwen]

Thanks for the link to the post. I have read it before, based upon the content and the date of the post, my impression was that it was mainly for FreeBSD. I thought one of reasons that TrueNas has been ported to linux, besides VM and docker supports, is that more hardware drivers in Linux. HP P440 could be a good example here because the manufacturer does support it in Linux. Besides the card doesn't have enough history in TrueNas Scale, is there any showstopper?

As has been pointed out, (sometimes hyperbolically , the issue with hardware RAID cards is not the OS, (FreeBSD or Linux), but ZFS. If ZFS has access to the storage devices through normal HBA ports, (aka SATA or SAS), then it can work properly in regards to data safety. Like write barriers, etc... Hardware RAID cards can hide such things. Plus, they can hide SMART access.

In fact, I have found that Linux is at times less reliable that other OSes, like Solaris, AIX or FreeBSD. It's partly that Linux requires modules to be re-compiled, even on minor kernel revisions. And partly that Linux has such a fast development cycle, their is zero chance any real production server can be on the latest kernel. (RedHat RHEL 7 still uses 3.x kernels!!!)

 

[HoneyBadger]

The key here isn't so much the hardware or the OS specifically, but the quality of the driver, given those two inputs.

And while I mean no disrespect to the developers of the ciss driver, LSI's is the gold/platinum/diamond standard for solidity.

There's exabytes of data and billions of aggregate run-hours of users with LSI drivers and hardware. It's a risk-avoidance issue, and long-term ZFS users tend to be very much on the side of "avoid risks."

 

[hyi]

Thanks for providing feedback. My use case is SOHO, the truenas machine would be placed in my office. it is not mission critical like most truenas users. Since the machine next to my desk, I prefer it would be quiet. Some HP machines don't like 3rd party hardware, and cooling fans would run at full speed with non-HP card installed. That why I thought about putting P440 into HBA mode instead of buying a LSI card.

The support for docker and VM in Truenas Scale is another key factor as it matches my use case. It is not because of that I am a linux fanboy as some comments suggested. I am OS agnostic, I have a pfsense running on FreeBSD, I run linux desktop, and run macOS for other workload.

I agree @HoneyBadger's comment on quality of the driver. My impression, right or wrong, is that there are more developers in linux community to patch hardware/firmware issues, so linux has more reliable drivers for some hardware/firmware, especially those buggy ones.

I fully understand that LSI is the highly recommended HBA for truenas. In the back of my mind, I prefer that LSI is not the only game in town. I like alternatives. Imaging, AMD is not here to compete with intel, ARM is not here to compete with x86.

 

[Ericloewe]

Hey, nobody likes monopolies around here either, but the sad reality is that most vendors just do not give a damn.

 

[hyi]

Should the community add the support first, or should vendors provide the support first? It is a phosphorus cycle. Twenty years ago, linux was in the worse situation than FreeBSD/Solaris. Anyway, I am very glad to see that TrueNas added Scale product line, and hope more hardware get supported.

 

[NickF]

The only non LSI Card I would even try for ZFS on SCALE would be the H240. It's pretty neat...SAS12 with 8087 connectors

Comparing HBA IT mode SAS controllers | 2020 Edition

This video is a follow up to my previous video "Comparing HBA IT mode SAS controllers" from 2018. Since then, my store has greatly expanded the selection of ...

youtu.be

 

[hyi]

Thanks @NickF. It seems that both H240 and P440 are gen9 controller based upon PMC chipset.

 

[Ericloewe]

Should the community add the support first, or should vendors provide the support first? It is a phosphorus cycle. Twenty years ago, linux was in the worse situation than FreeBSD/Solaris. Anyway, I am very glad to see that TrueNas added Scale product line, and hope more hardware get supported.

Well, the vendor knows their hardware, so they're the prime candidate to write a good driver. See Intel, LSI, etc.
That's not to say it must be perfect out the gate - it won't be and it's never been. See the endless updates to the mpt, mps and mpr stacks. Thing is, the vendor needs to give a damn. Back in the day, Highpoint I think it was had a fancy new HBA that looked promising and they produced a FreeBSD driver. A couple of folks jumped on that and started trying it out. It was unstable under real-world conditions and feedback was forwarded back to the vendor, followed by the deafening sound of crickets. This was a period of some six months, too.

 

[jgreco]

Anyway, I am very glad to see that TrueNas added Scale product line, and hope more hardware get supported.

Yes, lots more crappy hardware will be "supported", but just like all the crappy hardware that was already "supported", none of it will be a good idea to use. The fundamental problem is that there's shit-tons of crap-grade PC hardware out there. The OS cannot make fundamentally crappy hardware work better. And both FreeBSD and Linux already fully support the good hardware bits. You really do need to pick hardware that's going to work well, and that set is nearly identical between FreeBSD and Linux.