HBA Card Firmware

[Fastline]

Quality of the cables, connectors, backplanes - noise in the electrical signals caused by low quality "something".

Oh, this thing. Got it got it.

BTW, given the current state of hardware offers I would never try to build an U.2 or U.3 capable system from scratch but start with a Supermicro barebone with slots on front, backplane, connectors to mainboard ... all set and done. Just plug in your SSDs.

I know, those are very reliable and feasible but the thing is the space. When i have another big room, that is the plan for sure. Currently, the room is filled and i have no option other than this current machine. But yeah, will do that for sure for the features those nice racks offer :)

 

[Patrick M. Hausen]

I know, those are very reliable and feasible but the thing is the space. When i have another big room, that is the plan for sure. Currently, the room is filled and i have no option other than this current machine. But yeah, will do that for sure for the features those nice racks offer :)

But then why not just get a nice quiet "cube" style case combined with a mainboard with enough SATA connections like Supermicro's A2SDi series (up to 12 SATA ports) and use SATA SSDs ...

And if your priority is keeping your case and mainboard combo, still SATA is much more manageable, available, and probably cheaper when buying new.

As for your question about my U.2 SSD model: Intel P4510 1 TB. The firmware version is different for each line of SSDs so just comparing versions won't help much. But the latest firmware for any supported Intel/Solidigm drive can be flashed by booting the "SFUT" (Solidigm Firmware Update Tool, I guess) CD image. All info here: https://www.solidigm.de/support-page/product-doc-cert/ka-00099.html

 

[danb35]

I would never try to build an U.2 or U.3 capable system from scratch but start with a Supermicro barebone with slots on front, backplane, connectors to mainboard

...or look at something like a Dell R630/640/650 with the appropriate backplane and cabling.

 

[Patrick M. Hausen]

...or look at something like a Dell R630/640/650 with the appropriate backplane and cabling.

I have become a bit allergic to Dell after learning that they change properties of servers within what is supposed to be a standardised line and version - like swapping the network interface chips. And refusing any support even when you definitely have a hardware issue, when you are running an "unsupported" operating system. Combined with a "we ship drivers for Windows" attitude.

So, no, no Dell if I can avoid them.

Also but this is way back past - their IPMI console sucked rocks. And do they ship their systems with empty caddies for HDDs/SSDs now or still with dummies that you need to replace with real caddies?

Supermicro servers come with better than average (not perfect) IPMI and caddies that take your 3rd party drives and all screws and accessories you would ever need to add any component to their servers included. Clear win for me.

 

[danb35]

So, no, no Dell if I can avoid them.

Fair enough.

And do they ship their systems with empty caddies

No idea; the Dell systems I've bought have all been used--which also means their support (or lack thereof) is a non-issue for me.

Supermicro servers come with better than average (not perfect) IPMI

I haven't noticed that the IPMI on my X11DPH-T is notably better than iDRAC Enterprise on my R630. But I can deploy a TLS certificate to iDRAC with a couple of racadm commands, while deploying a cert to Supermicro's IPMI takes a pretty hack-y reverse-engineered script, so that's a point in Dell's favor (especially when using short-lived certs from my local CA).

 

[Patrick M. Hausen]

My iDRAC experience is admittedly a bit outdated.

 

[danb35]

What's odd to me is that my C6220 II has "Dell Remote Management Controller," while my R630 has iDRAC 8. The R630 is certainly a newer system, but iDRAC has obviously been around for quite a while if we were on release 8 at that time. No idea why they would have been maintaining two parallel remote management systems. But I guess I'm getting a little opposite-over-adjacent.

 

[Fastline]

...or look at something like a Dell R630/640/650 with the appropriate backplane and cabling.

Okay. I'll look into it.

 

[Ericloewe]

still with dummies that you need to replace with real caddies?

Dummies. Suspiciously-similar generic trays to the rescue.

have become a bit allergic to Dell after learning that they change properties of servers within what is supposed to be a standardised line and version

They don't seem to do this much, but knowing what to order is half the battle with them. For older stuff, pretty much all combinations of parts are widely available. Newer stuff has moved to OCP NICs, though HBAs are still highly proprietary (though the R6515/7515s use the same HBAs as Gen 13/14, which largely mitigated the problem).

Supermicro servers come with better than average (not perfect) IPMI and caddies that take your 3rd party drives and all screws and a

IDRAC 6 is painful in this day and age, but so is IPMI on Supermicro X9 boards, and those are a generation newer. IDRAC 7/8 are much better, with HTML5 iKVM (same basic software stack that AMI ships with MegaRAC). The interface is sluggish though, but not nearly as devastatingly slow as contemporaneous Lenovo servers.
iDRAC 9, which is in Gen 14/15/16, is a massive upgrade with a much beefier CPU.

They also actively support it much longer than Supermicro - Gen 12 got HTML5 around the same time Gen 13 did, and Gen 13 is still getting updates around a decade after launch.

What's odd to me is that my C6220 II has "Dell Remote Management Controller," while my R630 has iDRAC 8. The R630 is certainly a newer system, but iDRAC has obviously been around for quite a while if we were on release 8 at that time. No idea why they would have been maintaining two parallel remote management systems. But I guess I'm getting a little opposite-over-adjacent.

How different are they, though?

I understand that iDRAC originated from what AMI now sells as MegaRAC, so that accounts for at least some oddities.

 

[Fastline]

Well, never tried Dell IPMI but have used Supermicro and i have always found it easy to use. Definitely, i prefer Supermicro. Of course, i have started the NAS Journey and not that savvy like you guys ;)

 

[Fastline]

Guys, can i go with the normal consumer grade SSDs? I'm asking this for SATA and NVMe as well or do i have to stick to enterprise grade for latency, IOPS and high endurance?

 

[danb35]

How different are they, though?

The "Remote Management Controller" doesn't tie in to the OS at all the way iDRAC does (or at least can). It's a Java KVM vs. HTML5, though that could be a generation thing. There's no way I can find to automate deploying a cert to the RMC, while there is for iDRAC. The RMC doesn't seem to expose as much information about the system configuration as does iDRAC.

There are probably other differences, but these are the ones I've noticed.

Oh, and back to the "Dell vs. Supermicro" issue, the R630 (and I believe any later-generation servers) will update all the system firmware--BIOS, iDRAC, PSU, HBA, NIC, etc.--in one fell swoop, direct from Dell's website. Supermicro has no comparable feature.

 

[Ericloewe]

and back to the "Dell vs. Supermicro" issue, the R630 (and I believe any later-generation servers) will update all the system firmware--BIOS, iDRAC, PSU, HBA, NIC, etc.--in one fell swoop, direct from Dell's website. Supermicro has no comparable feature.

Yup, can confirm. Very useful, even though the feature is somewhat hidden in iDRAC 8 - in 9, they added a checkbox for Dell's default firmware server, so it's most obvious that there's no setup required.
The only trouble I had with it was when I needed to update an R720 from close-to-release firmware all the way to a decade later. For various reasons, I couldn't get it to work very well via the traditional "upload a file" option, but HTTPS support wasn't implemented yet, the FTP and whatever else servers had all moved years ago and dropped Gen 12...

 

[nabsltd]

What do you think about these two?
10Gtek S94N2X8
10Gtek S94N416

Not even for a 100GbE network setup? BTW, what kind of storage do you recommend for a 40GbE/100GbE?

10Gtek has solid hardware. I've never been disappointed by anything from them. They aren't the cheapest, but you won't go wrong.

Don't worry about faster Ethernet until you have enough active clients to make 10Gbps too slow. You can run dozens of 1Gbps clients with 10Gbps storage. You can also separate traffic on VLANs/subnets so that a storage server with 4x 10Gbps cards handles 20x 1Gbps clients on each 10Gbps link. But you don't even need to do that. I have well over 100 machines at my work that have 1Gbps and connect to a server with only a single 10Gbps NIC. The trick is, even if every machine is being used, they are only reading/writing about 50Mbps on average over the day.

 

[nabsltd]

Short version: for Intel drives always make sure to run the latest firmware. The SSD division has been spinned off (or sold?) as "Solidigm", so that's where you will find the update tools now.

The Solidigm Storage Tool has support for enough Intel branded drives plus the newer Solidigm drives and they don't hide it behind a registration...just download. Most other enterprise SSD manufacturers treat their drives a "OEM-only", so they expect you to get support from somebody else.

 

[Fastline]

10Gtek has solid hardware. I've never been disappointed by anything from them. They aren't the cheapest, but you won't go wrong.

Don't worry about faster Ethernet until you have enough active clients to make 10Gbps too slow. You can run dozens of 1Gbps clients with 10Gbps storage. You can also separate traffic on VLANs/subnets so that a storage server with 4x 10Gbps cards handles 20x 1Gbps clients on each 10Gbps link. But you don't even need to do that. I have well over 100 machines at my work that have 1Gbps and connect to a server with only a single 10Gbps NIC. The trick is, even if every machine is being used, they are only reading/writing about 50Mbps on average over the day.

Great. Thanks for your input!

 

[Fastline]

Guys,

Can anyone suggest if the Samsung 870 EVO/QVO can be a good drive for data vdev for TrueNAS?

 

[Ericloewe]

Guys,

Can anyone suggest if the Samsung 870 EVO/QVO can be a good drive for data vdev for TrueNAS?

If they're reliable these days. Big if.

 

[Fastline]

If they're reliable these days. Big if.

So, consumer SSDs are not good or this specific Samsung 870 EVO/QVO or the Samsung itself?

 

[Patrick M. Hausen]

So, consumer SSDs are not good or this specific Samsung 870 EVO/QVO or the Samsung itself?

Samsung "prosumer" lines like "860 Pro" or "970 EVO Plus" have been very reliable for us in the past. We still have a lot in operation without any issues.

So no, not generally "not good". The new "QVO" line has been found by various reviewers to suffer from performance degradation due to the internal architecture. I am not current with respect to the state of controller firmware updates that might have fixed these issues.

You will probably have to do a little bit or research yourself - I'd start with e.g. Serve The Home. IMHO the most important metric is the guaranteed write endurance (TBW/PBW) value followed by any performance considerations.

In general you best go with a reputable vendor and I do not consider Samsung specifically unreliable - far from it. But as this market is moving so fast there really is no answer to questions like this that will not be outdated in another six weeks or so.