Pedants Take on the great Jellyfish fry-fest

[Constantin]

/rant on (with apologies, but this video was a bit egregious)

So, @Joon posted a Linus TechTips youtube link, where Linus allegedly rips the Jellyfish from OWC to shreds using TrueNAS, etc. Some observations:

• It's very odd to be using a cheaply-built $950 iStar case for a production server. I'd argue that a used SuperMicro 846 with dual PSUs ($400 on eBay) would perform better at a much lower cost.
• The two-supply-in-a-ATX case is a neat engineering solution to a problem that the 846 doesn't have. @jgreco's likely would take issue when Linus proudly trots out a Twins Pro 700W, allegedly redundant, PSU for a 24-drive (20HDD, 4 SSD) server. Per @jgrecos guide, the peak load can max out both PSU's, eliminating their redundancy...
• I'd prefer the SuperMicro 846-included dual 80+ Gold 1,200W PSUs simply on the basis of wider availability on the new and used market.
• Want a higher-efficiency PSU than the 80+ Gold offered by the Twins or the stock SM PSU? No problem, you can buy two new 1,200W PSUs directly from SuperMicro that are Titanium rated and still spend less than Linus did for the Twins Pro 700W ATX assembly.
• Plus, the Supermicro PSU and case actually features a good backplane design that plugs straight into their PSU as opposed to the Octopus-fest that Linus seems to be enjoying with iStar in the video.
• The fan choices also just make me shake my head. There is a reason that iStar isn't on my buy list and the stock fans inside the SuperMicro actually prevent the HDDs + CPU from stewing in their own juices.

So, in my opinion, Linus' case choice + PSU will cost hundreds of dollars more and perform worse than what is recommended in the hardware resource guides that @Chris Moore and @Eric Loewe put together. Presumably, sponsorship had something to do with Linus' odd case and PSU choices. As for the rest of the NAS...

• The HDD choices seem solid. He drives are a great choice for something as dense as that server.
• The Sabrent 2TB NVME choice for a SLOG is pretty confusing considering the likely performance of same. The capacity way exceeds what he needs for a SLOG. With all the money one saves buying a used 846 case with its PSUs ($1,100) and the Sabrent 2TB drive ($429) one would be able to put in a $1,300 375GB p4801x and really go to town. Adding a second SLOG in a mirror would make sense, however.
• The board, CPU, 2TB L2ARC for a 256GB RAM / 244TB Pool seems about right also.
• The LSI 9305-16i is a good choice for an HBA.
• A boot pool with 240GB drives means he won't ever run out of SWAP or system update image room.

Anyhow, I get how he wanted to replicate the DAS-like feel of the Jellyfish by inserting a lot of 10GbE NICs but if his point was to outclass the Jellyfish, he kind of misses the mark. I doubt that a 2-VDEV, 20-drive Z2 pool using sync=on will be able to consistently max out a single 10GbE connection, especially with that SLOG.

For someone who consistently tries to needle folk for spending too much money for allegedly overpriced solutions (i.e. Jellyfish), Linus is remarkably off the mark in this video re: what would make a sensible case, SLOG, and power supply selection.
/rant off

 

[HoneyBadger]

I haven't watched the full video yet (and I'm getting some deja-vu, didn't he already do a "Jellyfish Fryer" video months ago?) but I'm betting he wanted to avoid the cries of "that's unfair, you can't compare used hardware to new" if he opted to pick up a SC846 off eBay.

I'd still opt for the SuperMicro case though for all the reasons you stated (PSU, fan, backplane)

Edit: Skipping through it, they did an all-SSD Jellyfish fryer before. That's what I was remembering.

SLOG isn't write cache, and it won't even get used on the SMB share from Windows unless you force it. Once you do, the Sabrent is a bad choice as highlighted already by @Constantin - a pair of the Optane P4801Xs is way better. Might even be able to do it with the 200G models as the sequential/large block speeds aren't too much of a loss from the 375G models and they're about 60% of the cost, so you could go mirrored with the onboard M.2 slots and punt L2ARC to a PCIe carrier card (even a PLX switched one would probably be okay for L2ARC purposes) Scratch that, the EPYCD8-2T apparently supports PCIe bifurcation to x8x8 or x4x4x4x4 on PCIe slot 1, and x4x4 on slot 4. Get a dual or quad-slot carrier card and go to town with whatever NVMe you want.

Question for the TrueNAS team - have you guys considered pinging Linus to see if he wants a hand with a little fine-tuning, in a sense of "You got it to go fast for cheap, let's go even faster for free"

Edit 2: After reading through the YT and LTT forum comments, I'm really wanting to write a long-winded explanation of how SLOG actually works because I still see people making mistakes and assumptions based on how it used to work.

 

[Constantin]

Fair point... I was comparing our resources-suggested approach vs. his build.

However, the Supermicro CSE-846BE1C-R1K23B is available for $1500 new from WiredZone, which is about what he paid for the iStar, Noctua, and Twins Pro PSU combination. Difference being a higher-capacity PSU that also features significantly better efficiency at part-load, super-micro case and PSU build quality, redundant cooling for the CPU.

Granted, going all new re: the case will cut down the available budget for the SLOG but I'd wager that a 100GB p4801x will likely do better in a SLOG application at the same price point as a Sabrent 2TB general-purpose SSD. Like you, I'd try to opt for the 200GB model since it offers much better performance at smaller block sizes. Ideally, in a 2-mirror pool using a PCIe 3.0x4 m.2 connector.

I do wonder if the CPU isn't hilariously over-spec'ed if the sole purpose of this server is DAS-like fast storage. Maybe that's how one could use the same overall budget to invest in a better or mirrored SLOG. Still, I am a fan of big boards with lots of PCIe slots simply because it provides flexibility as the use case changes. Be pretty funny to have just a SLOG pool cost the same as the motherboard, CPU, and RAM.

 

[HoneyBadger]

Still, I am a fan of big boards with lots of PCIe slots simply because it provides flexibility as the use case changes. Be pretty funny to have just a SLOG pool cost the same as the motherboard, CPU, and RAM.

I have a feeling that's why EPYC was specced here. Tons and tons of lanes for PCIe slots and onboard M.2 NVMe.

Although the vertically oriented cooler gave me another "why" moment.

 

[Constantin]

Although the vertically oriented cooler gave me another "why" moment.

Another opportunity for highlighting a sponsor? (Noctua)

 

[HoneyBadger]

Another opportunity for highlighting a sponsor? (Noctua)

I've got no beef with the cooler choice itself, just the orientation - the fans should be blowing front-to-back towards that nice empty exhaust grille, not bottom-to-top sending the CPU TDP into the PSU intake.

 

[Constantin]

Agreed, except the locations of the fans vis-a-vis the plate that mates to the CPU appear to be fixed. So I doubt it can be changed. a 90* turn would help expel the air. In a tower, this heat sink orientation makes a lot of sense if you have one or two fans in the top of the case.

I wonder to what extent the performance of a good passive copper cooler would be roughly comparable to this active one with the plastic supermicro air shroud in place - if you look at the wired-zone listing, the air shroud interfaces with 2 or 3 fans in the front and 2 in the back, pushing/pulling air across the non-PCIe-slot regions of the motherboard.

 

[Constantin]

I also wonder to what extent the EPYC choice was driven by the lack of Ryzen motherboards and chips. Few gamers / miners choose EPYC.

 

[danb35]

Linus is the same guy who drills holes in motherboards that don't fit, and is then surprised when they don't work any more. Really not a good look for iX to be doing anything to support or recommend him.

 

[Constantin]

I want to think that iXSystems would be nothing more than a innocent bystander. However, given Linus’ shilling of various enclosures, power supplies, etc. its likely that much in that video is a paid-placement advertisement.

for example, he mentions that the drives were shipped from Seagate to him.... but Linus’ operation is not “big” enough to be a data-center size B2B client getting direct drop shipments from a OEM. Those drives should be coming from a distributor or merchant like Amazon, wiredzone, Newegg, whatever.

Nope, as phrased, those drives likely came straight from the influencer division within the Seagate marketing department. I’d expect no less re: the cheap case from iStar, or the laughable Twins power supply.

 

[jgreco]

• The two-supply-in-a-ATX case is a neat engineering solution to a problem that the 846 doesn't have. @jgreco's likely would take issue when Linus proudly trots out a Twins Pro 700W, allegedly redundant, PSU for a 24-drive (20HDD, 4 SSD) server. Per @jgrecos guide, the peak load can max out both PSU's, eliminating their redundancy...

Can we please not link to the clickbait videos of YouTube morons?

(Not a moderator request, but rather a personal wish).

 

[Constantin]

FWIW, I didn’t directly link to the video in question but rather the forum posting from iXSystems’ own @Joon making us aware of it in the Announcement section. (See above) However, the @Joon thread was closed to further comment... hence I started a new thread in the off-topic section of this forum.

Perhaps it was my reaction as well as yours that explains why the @Joon thread was closed proactively? Other users seem to have liked it based on all the positive feedback received.

 

[HoneyBadger]

Potential hot take incoming.

Linus is the same guy who drills holes in motherboards that don't fit, and is then surprised when they don't work any more. Really not a good look for iX to be doing anything to support or recommend him.

I've got the opposite opinion. When Linus is making a build that leverages and/or heavily uses their product, it's in their best interests to help optimize and deliver a good solution. This isn't a "Top Gear of technology" kind of meme-video, like immersion-cooling a server in a bathtub of mineral oil, but rather something that has practical use and runs a lot closer to a traditional comparison test. Even if you have a particular beef with someone, it's good to reward when you see things done well (even if we're all looking at it and discussing diplomatic ways to phrase "we can help you do it even better.") Mistakes might have been made, but they were born of unfamiliarity, not ignorance. Heart in the right place, good intentions, and all that. So let's help make sure the path is getting paved to the right end goal.

Regarding the whole "clickbait" aspect - that's unfortunately part of the deal with YouTube. Several prominent content creators have dug into this, but the end result is that the "face closeup, bright colours, giant text" typical "clickbait" thumbnails get significantly more engagement than others. Blame for this one rides squarely on the viewing audience, sadly.

 

[danb35]

When Linus is making a build that leverages and/or heavily uses their product, it's in their best interests to help optimize and deliver a good solution.

His video persona, at least (I can't speak to the underlying person, of course), is a moron who does stupid and destructive stuff, including with FreeNAS (IIRC, it was some insane number of drives in a hardware RAID6) in the not-too-distant past. The most appropriate response from iX is, "we disavow any connection with, or support of, this moron. We do not now have, nor have we ever had, any association with him. Here's how to do this right." His level of on-screen stupidity is so extreme that I'm unwilling to believe it isn't deliberate--so in that sense I agree with you that the mistakes weren't born of ignorance. Of course, the obvious problem with that is that being an idiot on video is popular, so he has a large audience, many of whom are even more ignorant than he appears to be and therefore might think he has some idea what he's talking about.

Unfortunately, it looks like iX are doing neither what I'd recommend (publicly disavowing him) nor what you'd recommend (trying to educate him), but instead celebrating that they got screen time on his channel. Rather short-sighted, I think.

 

[HoneyBadger]

While I'm no fan of the "play the fool for clicks" content either, we'll have to agree to disagree on the correct method of approach here I guess.

 

[Constantin]

As a community, I’d like to think that we’d put more thought into the planning of that rig vs. the actual build. That is, explain what you intend to build, and why you’re choosing the components you picked.

for example, a pure storage rig for fast editing would likely benefit from using

• three 8-disk Z2 VDEVs in a pool (Ie fill all those HDD slots) rather than two VDEVs. (“pay” for the extra 4 disks by buying a used SC846 @ $400 vs. spending $1,500 on the new case and PSU). Spend another $250 on a spare PSU from SM if you need the insurance
• A more suitable SLOG if sync has to be on. (Ie 200GB p4801x). You’d be able to buy 2 and mirror them if you go used on the motherboard, CPU, and RAM like we recommend in our hardware guides.

The role of the social media advisor here is to increase the profile of iXSystems but I wonder if that announcement could not have contained some additional tips like the above to put the rig in context. (Ie “here is what we’d recommend from our lineup as an alternative”, and why).

it would also be great to know if iXSystems paid for the product placement as it would put the announcement in better context.

 

[HoneyBadger]

Social media and PR releases generally need to be non-confrontational and polite. Vendor-on-vendor can certainly get a little spicy (see the Dell/HP/IBM/etc world and how much mud they like to sling at each other's solutions!) but when engaging with content creators or review sites, it's always best to be diplomatic, cordial, and always assume the best intentions.

Think something like:

Hey Linus,

Thank you for featuring TrueNAS CORE as the software of choice in your latest "Jellyfish Fryer" video (URL embed). We put a lot of hard work into revamping and streamlining the user interface and overall experience with the 12.0 release, and our dev team was glad to hear your positive feedback. We'll keep improving and refining it as we go forward.

Since the TrueNAS system won the race, would you be interested in a quick session with one of our (engineers/consultants) to take a victory lap? We spotted a couple sections where we believe a little fine-tuning would widen that price-to-performance gap even more.

Feel free to reach out to me via (email/telephone/social media/carrier pigeon) if you have any questions.

Thanks again!

iXsystems

(Obvious disclaimer: I am not an iXsystems employee, this isn't an actual letter, but if you're reading this Linus, my DMs are open. ;) )

 

[Constantin]

I agree. It's a win-win especially when a solution can be made even better with minimal investment. It also highlights the capabilities of the sales engineers at iXsystems, i.e. understanding the use case and then proposing a solution that fries the jellyfish better, faster, and at lower cost.

Key thing here is the knowledge/dedication/competence that iXsystems can bring to customers as part of the planning, sales, configuration, and ongoing maintenance / repair experience associated with their IT infrastructure. That's the key differentiator vs. competitors since the hardware can be purchased by anyone.

If I had CIO responsibilities, I'd want iXsystems in my corner based on my experience dealing with their customer support.