LSI MegaRAID 9460-16i with physical drives in JBOD mode

[jgreco]

A developer that can't master a simple programing language (C++), is probably not the best example of a developer.

C++ has been called many things, but "a simple programming language" isn't one that I've seen associated with it.

There's "weird mess" / "really terrible, terrible language" --

Jonathan Blow: "C++ is a weird mess"

Sometime around 2014, while still developing The Witness, Jonathan Blow sat down to work and realised what a miserable …

www.gamesindustry.biz

Google says in response to "Is c++ easy" --

Is C++ easy?
Is C++ hard to learn? C++ is known to be one of the most difficult programming languages to learn over other popular languages like Python and Java. C++ is hard to learn because of its multi-paradigm nature and more advanced syntax.Jan 18, 2022

Etc.

So I have to wonder if you're just trolling here.

 

[Whattteva]

yes, lots of people rant about assembly language being the best when I first started programming.

Ok... this doesn't prove anything. His position STILL hasn't changed even after all these years.

A developer that can't master a simple programing language (C++), is probably not the best example of a developer.

Oh yes... I suppose Linus Torvalds isn't the best example of a good developer... Perhaps, you could rewrite/maintain the Linux kernel to a far more superior state than him in C++.

To quote this thread "CPython 1.0 was released in 1989. At that time, C was just recently standardized. C++ was almost unknown".

in any event, this thread was about HBA good, and RAID JBOD bad.

It's funny that you quote that thread and just cherry-pick the part that fit your narrative.

It goes on to say:

Linus Torvalds's rant against C++ was brought up, so I'll mention that as well. Nothing I've seen from Guido indicates that he has that sort of strong, negative feelings toward C++. About the worst I've seen him say is that teaching C++ is often a disaster -- but he immediately went on to say that this is largely because the teachers didn't/don't know C++.

I also think that while it's possible to convert a lot of C code to C++ with relative ease, that getting much real advantage from C++ requires not only quite a bit more rewriting than that, but also requires substantial re-education of most developers involved. Most well-written C++ is substantially different from well-written C to do the same things. It's not just a matter of changing malloc to new and printf to cout, by any stretch of the imagination.

Note that he says even the teachers didn't know C++. Moreover, it says that rewriting will require substantial re-education and he's talking about the Python developers, which I'd imagine would know a thing or two and not just some "bad" developer that you seem to keep hinting.

Also, a language that allows you to do things like Multiple Inheritances isn't exactly what I'd characterize as a "simple language". Very few modern languages even allow it (can't even think of one besides C++) due to how easy it is you can come across the "Diamond problem" issue and get an undefined behavior.

 

[john60]

They'll likely load without issue. That was never in serious doubt; it's what happens during exceptional conditions that's really the problem. It is unlikely that you will reach the year mark and still have it be problem-free (possibly sooner).

Fear is a powerful tool. You yield this tool eloquently.

I'm the kid that grew up in a house where the bathroom faucet was grounded and the chain on the light switch above the sink was
live. As a young kid, I quickly learned to fear that chain. Later, I learned the protocol to be not electrocuted. Even later with some education,
I learned how dangerous it could be and I fixed the light so others in my family would not get hurt. Thank you for trying to protect me, but I want to touch that chain one more time.

Lots of people have had problems with the MFI firmware and driver. If you're lucky, you'll be on the MRSAS driver and MFI firmware, but still suffers some of the same underlying issues.

mfi0: COMMAND 0xfffffe0000ef TIMOUT AFTER on Dell R730xd

Hi folks, Back in November I bought 2 x Dell R730xd systems due to the fact they had a lot of drive bays and the disks could be operated in JBOD mode to maximize ZFS's abilities. I installed and deployed them with FreeNAS-9.3-STABLE-201509022158 They have been running fine ever since, until...

www.truenas.com

or search for "mfi0" and "timeout".

The referenced thread is about
1) support from DELL. I'm betting DELL would also not support truenas on HBA.
2) this was not a hw issue?
3) no mentioned that the disks were not readable by the repaired system, or did I miss this
4) points to freenas hw requirement says you can use RAID in a pitch, but watch out for:

• Could mask disk serial number and S.M.A.R.T. health information
  Presumably, if raid 0 is the implementation mode for JBOD
• Could perform slower than their HBA equivalents
  Presumably, a throughput test can validate this
• Could cause data loss if using a write cache with a dead battery backup unit (BBU))
  how is this any different from cache on disks and SSD? Maybe because it spans multiple disks?

I like your quote in this tread "It's kinda like using the microwave to dry your clothes.", I understand hospitals do this to dry and sterilize linen. So only a bad idea if the power is too high and your clothes catch fire.

My system has not arrived yet. Do you know who makes the 9361-8i card and how does MFI relate?

 

[john60]

C++ has been called many things, but "a simple programming language" isn't one that I've seen associated with it.

There's "weird mess" / "really terrible, terrible language" --

Jonathan Blow: "C++ is a weird mess"

Sometime around 2014, while still developing The Witness, Jonathan Blow sat down to work and realised what a miserable …

www.gamesindustry.biz

Google says in response to "Is c++ easy" --



Etc.

So I have to wonder if you're just trolling here.

Not trolling, allow me to clarify.

Once you master programming or even partially masted programming, you can write a lot of programs.
Many people can program in a boatload of languages, C sharp, C, lisp, prolog, protel, java, javascript, C++, etc

Even knowing a bunch of these, you are not ready to program networking protocols, wireless signal processing, phase array radar, non-stop telecom system, etc. Designers in these systems are well-passed thinking C++ is complicated. At least that is my experience.

I would have thought that RAID cards would be programmed by the latter group, but I could be wrong.

In any event, I don't want to debate programming languages. I am looking for details of things to watch out for with my 9361-8i card.

 

[jgreco]

Fear is a powerful tool. You yield this tool eloquently.

I've been faced with many hundreds of users with RAID card problems. I lean little-l libertarian, which among other things means that I believe you're allowed to earn your lumps the hard way if you like. However, I also think it's the decent thing to do to save you the time, trouble, cost, and data loss of warning you.

I want to touch that chain one more time.

Be my guest.

Do you know who makes the 9361-8i card and how does MFI relate?

LSI, Avago, or Broadcom presumably make the 9361-8i, which is their manufacturer product number. OEM cards manufactured by LSI, Avago, or Broadcom might also carry this number, but more usually gain a specific OEM part number like ServeRAID M5210.

MFI relates because it is the conventional name for both the MegaRAID Firmware Interface firmware and driver. It was sufficiently sucky and deficient that LSI rewrote the host-side driver for FreeBSD 10.1.

 

[Whattteva]

Once you master programming or even partially masted programming, you can write a lot of programs.
Many people can program in a boatload of languages, C sharp, C, lisp, prolog, protel, java, javascript, C++, etc

Being able to program in a lot of languages != you're a master at those languages. I started with Java in college, moved on to a C/C++/C# in my first job, and then to Objective-C and Swift in my current position. Sure I can make some programs in C/C++/C#/Java still, but my expertise remains in Swift because I have stopped using those languages at least 6 years ago. Yes, I know the basic architecture/design patterns, but that doesn't make me a master in those languages that I haven't been using for so long, especially since C++ is like a moving target these days as they keep expanding the standard... I don't even know what they're on anymore now C+21?

Even knowing a bunch of these, you are not ready to program networking protocols, wireless signal processing, phase array radar, non-stop telecom system, etc. Designers in these systems are well-passed thinking C++ is complicated. At least that is my experience.

You must be kidding... I was a driver developer in Lockheed Martin for power and RF systems. Those domain knowledge is certainly important and complex in itself, but that doesn't mean I all of the sudden think C++ is not complex just because what I'm coding is complex also. It makes your mastery in the language itself all the more important. I wouldn't in a million years hire someone who's primarily been writing Java web apps to write C drivers for mission-critical US Navy systems.

 

[jgreco]

Designers in these systems are well-passed thinking C++ is complicated. At least that is my experience.

I would have thought that RAID cards would be programmed by the latter group, but I could be wrong.

In my experience, and bear in mind that my background includes medical devices intended for vital statistics monitoring use in operating rooms, I found that working with embedded systems and RTOS required a very particular skill set, highly experienced people capable of writing optimized C or assembly code, and not ruined by the more abstract object oriented or interpreted languages that hide so much of the details of what's going on with the hardware.

In college, I was the guy who watched idly in the VAX assembly class I had to take, despite previous assembly/ML experience, was given a set of maybe a dozen instructions, and was told that this was the shortest non-specialized-CISC instruction set subroutine to implement some sort of copy. I sat back for about a minute or two, and corrected the instructor with a subroutine shorter by one instruction. Loop structuring error.

The effort needed to program efficiently at a low level favors people who grew up writing assembly and/or low level C; some of us actually understand what's happening with the program counter, stack, registers, etc. because we have done years of work down there. When I look at C, I see pseudoassembly. Most of the junior devs these days have a hard time coding credible Java or Python, and have no clue what's going on under the sheets. I wouldn't want someone coding for an RoC chip whose sole claim to experience was their uber eleet JavaScript skillz.

 

[john60]

I played with JBOD, and it was fun. I decided to follow @jgreco and retire the RAID card and replace it with a $140 Supermicro 9300-8I 12Gbps HBA IT mode ZFS FreeNAS unRAID+2*SFF-8643 SATA US from ebay. Like I said before, fear is a powerful motivator. I figure that $140 was a small price to pay to simply future debugging.

 

[anthony11]

No, you cannot switch the personality of the controller to make it act as a true HBA - not interested in arguing the point, it's just a fact

You're confusing the nomenclature. Anything that bridges the PCI-e bus to SAS/SATA is an HBA. Host Bus Adapater. That's true whether or not it contains an RoC.

The best answer is to buy NVMe-ONLY chassis and avoid HBAs -- especially LSI RoC models -- altogether.

 

[Ericloewe]

You're confusing the nomenclature. Anything that bridges the PCI-e bus to SAS/SATA is an HBA. Host Bus Adapater. That's true whether or not it contains an RoC.

That's a little excessive in terms of pedantry...

The best answer is to buy NVMe-ONLY chassis and avoid HBAs -- especially LSI RoC models -- altogether.

I'm with you the minute NVMe HDDs become a thing. Until then, SAS and SATA still have a place, despite their many annoyances.

 

[Patrick M. Hausen]

That's a little excessive in terms of pedantry...

Isn't that a core competence of most tech communities? He who is free from pedantry throw the first stone ...

 

[anthony11]

Hah. Pedantry or not, it's the truth. And misusing the terms leads to people not understanding what the components are and doing things like ordering *both* an IT and an IR type HBA for a given system.

>I'm with you the minute NVMe HDDs become a thing. Until then, SAS and SATA still have a place, despite their many annoyances.

SAS is mostly dead. We won't see many new SAS SSDs released, and SATA SSDs are already disappearing from roadmaps.

Why would you *want* an HDD? I'm suspect you think that they're cost-effective, though in many cases that's no longer true.

 

[jgreco]

You're confusing the nomenclature. Anything that bridges the PCI-e bus to SAS/SATA is an HBA. Host Bus Adapater.

I'm not confusing the nomenclature. A SATA AHCI controller or a PCH SCU controller is not generally referred to as an HBA. Additionally, a RAID controller might be considered an HBA by your definition, but is certainly not considered one around here. Your method is like calling everything with wheels a car, even if it happens to be a a truck, railcar, tricycle, or dune buggy. If we wanted pedantry, an "LSI HBA" is actually a microcomputer trapped on a PCIe board. What we mean when we say "HBA" is generally shorthand for "LSI HBA", which you're correct is probably artificially limited nomenclature, as there are a few other HBA manufacturers such as PMC Sierra and Highpoint. But we still mean "LSI HBA" and not any of those others because what TrueNAS needs is a proven HBA with specific operating characteristics, and that turns out to be ONLY LSI HBA's. Since LSI HBA's are by a huge margin the most common HBA's, we do get tired of typing "You need an LSI 6Gbps HBA based on the 2008 or 2308 silicon, or the LSI 12Gbps HBA based on the 3008 silicon" and we do abbreviate that down to "HBA" because the longer form is so frickin' inconvenient to type a hundred times a day. This matches up with the definitions of most, but not all, manufacturers in describing their products.

 

[Ericloewe]

Why would you *want* an HDD? I'm suspect you think that they're cost-effective, though in many cases that's no longer true.

In many cases it still is, and will be unless some major shift in either the SSD or HDD markets happens. For 400 bucks, I can either buy a 20 TB HDD or a 4 TB SSD. If I don't need SSD performance and don't have other constraints, the math is clear.

 

[jgreco]

doing things like ordering *both* an IT and an IR type HBA for a given system

Which is perfectly fine and may even be recommended in certain cases. But "IT" and "IR" are not types of HBA, they are firmware classes for LSI HBA's. I run more than a dozen TrueNAS systems with both an IT and IR HBA in them.

 

[Scharbag]

In many cases it still is, and will be unless some major shift in either the SSD or HDD markets happens. For 400 bucks, I can either buy a 20 TB HDD or a 4 TB SSD. If I don't need SSD performance and don't have other constraints, the math is clear.

Exactly - we run Alletra SANs at work and they are hybrid. Spinning rust is not going anywhere soon given the price difference. As was said, you do not always need SSD level performance so why pay the high price?