DIY TrueNAS Take Two

[softwaremaniac]

Hello again!

About 18 months ago, I posted here about my wishes here and you made me realize that I had the completely wrong idea of how everything should be done.

The goal is to create a system that I can expand later as I want to. Primary use will be archival/cold storage and filesharing.

I've grown a lot and learned a ton in the meantime.

My current config plan looks like this:

Asus WS C422 PRO/SE
Intel Xeon W-2135
Noctua NH-D14 + LGA 20xx mounting kit
128GB DDR4 EEC RAM (8x16GB)
2x Kingston A400 SSDs for the install and maybe one cache drive. I know there are better choices, this was best bang for the buck at the time of the purchase.
1x NVME drive for cache
Corsair HX1200
2x LSI 8-port HBA cards
Fractal Design Define 7 XL + extra brackets

I will probably start with 8-10 drives and expand down the road.

The plan is to put them into 2 vdevs, 5x20TB and 5x16TB with RAIDZ1 with room to expand down the road if I want to add another vdev.

I have all the parts but the drives and will be buying them soon.

Except the archival storage I will likely try to setup Nextcloud.

Any suggestions welcome!

 

[Patrick M. Hausen]

What do you intend to use the NVMe drive for? With file sharing via SMB an SLOG will never get used.

 

[softwaremaniac]

I was thinking cache drive for faster performance.

 

[Patrick M. Hausen]

There is no such thing as a "cache drive" in ZFS although one could describe L2ARC as such.

SLOG is not a cache and is only used for synchronous writes.

L2ARC might be useful for reads, if all your RAM is used for ARC and you still get ARC misses.

Simply adding some SSD and expecting "something" to get "faster" won't lead to the desired result.

So again: what exactly do you intend to "cache"?

 

[softwaremaniac]

Based on my use case it does not look like I would benefit much for an NVME.

 

[softwaremaniac]

Great news! The server has been built!

I only have one problem. My 10G SFP+ card is not being detected.

The same card works in Windows, so I'm guessing some kind of incompatibility.

Will put in my Intel X550-T2 in next week and test.

 

[WN1X]

Which 10gig card were you trying to use that was not recognized?

 

[jgreco]

My 10G SFP+ card is not being detected.

Yeah, there are lots of tragic, terrible, and/or unsupported choices for "10G SFP+ cards". It's kinda like saying "My HDD is not being detected" without giving any other details. Please do feel free to check out the 10 Gig Networking Primer for information on well supported options, and also this article to explain why that X550-T2 may be a bad option.

 

[softwaremaniac]

The card is: PLT-ENW-9801

I've tried several Maxlink SFP+ cables.

 

[jgreco]

Mmm.

https://www.planet.com.tw/storage/products/49493/EMQ-ENW-9801_v3.0.pdf

A sketchy off-brand Taiwanese manufacturer of "upscale" PC parts.

I was interested a bit when I saw the claimed feature set. That, combined with the PCIe 2.0 makes me suspect that it's an Intel EXPX9501AFXSR knockoff from around 2007, when Intel started to get pissed off and tweak their driver to be incompatible with knockoffs. But I'm pretty sure it's Intel cloned silicon. Possibly 82598EB. It's gotta be in that window I think because of the PCIe 2.0... but Intel ARK doesn't seem to list the very old parts anymore.

 

[jgreco]

Hey @Ericloewe you sometimes have better luck than I locating obscure documentation...

 

[Ericloewe]

Hmm, they claim Intel and that manual seems to have been drafted circa 2018. The PCB is clearly not standard Intel and seems generally emptier than 8259x PCBs tend to be. If the manual's to be trusted, it shows up as an X520, so an 82599 controller or knock-off thereof. Since they sell it as a single-port card, I suspect these are grey market rejected ICs.

That said, I'm much more fascinated by their 1GBase-T cards, which claim to be I350s and support 802.3at PoE. I'd never heard of such a thing and I almost want one just to see if it's real and if it works.

 

[jgreco]

Since they sell it as a single-port card, I suspect these are grey market rejected ICs.

Could be. Noted: PCIe x4. Kinda like someone to make a workstation/desktop knockoff card.

Either way, my fuzzy memory says there was a time when there were a bunch of common Intel "clone" knockoff cards and then one day Intel played hardball and locked down the driver. I'd say late ought'ies. It was during that era when 10G was still relatively uncommon outside the enterprise realm. What I had been hoping to find was a driver download page, because I think some of them worked around it by hacking the driver to bypass the validation check. You know, similar to the whole SFP+ validation check.

 

[Ericloewe]

Some of their other cards do advertise a driver for download, and the manual for this one does speak of getting a driver from them, supporting your hypothesis.

Maybe they transitioned from dodgy silicon to slightly-less-dodgy silicon and dropped the hacked driver?

 

[jgreco]

Maybe they transitioned from dodgy silicon to slightly-less-dodgy silicon and dropped the hacked driver?

One would expect, then, if they could drop the hacked driver due to better silicon, that the silicon that worked then would also work now. Unless Intel was playing cat and mouse and newer versions of the driver got progressively better at fake detection. Maybe they are distributed with a driver CD, and no one worries about keeping up with OS updates?

This reminds me of the crap that went on with the Prolific PL2303 and FTDI serial-to-USB chips, where at least one of the manufacturers released a firmware that would unrecoverably brick the fake ones (conveniently found at EEVblog, along with a comment in thread about Drobo using the Prolific fake chips for something or other).

In any case, I noticed that the OP's card only seems to be sold in... well, I believe the term is "second world countries", or former eastern bloc/Warsaw Pact/China/etc, with lots of webshops in .hr (Croatia), .tw (Taiwan), .sk (Slovakia), and .ru (Russia) as reported by Google Search for the search: PLT "ENW-9801" The search result is so weird that I don't really understand it. Which isn't shocking since I'm mostly familiar with the US distribution channel. But if these things were any good, I'd expect to see them available in other countries. The fact that they aren't is incredibly suspicious and I'd sorta like to understand it better (which probably explains why I spent a half hour digging).

 

[softwaremaniac]

So, if I understand you correctly, my best best is in fact to replace this with something reputable just as I planned.

And yes, I am in Croatia.

 

[Etorix]

Indeed, "something reputable" is advisable. Solarflare NICs sell for under $50 and, for good or bad, come fast from Shenzhen to the EU through the "New Silk Roads". I've never heard of Solarflare or Chelsio knock-offs; this worrying issue appears to be confined to Intel NICs and LSI HBAs.

Solarflare SFN7122F DUAL-PORT 10GBE SFP+ Adapter SFN7500 | eBay

ETA 3 to 8 busines.

www.ebay.com

(no endorsement of the specific seller, I just picked up an example)

 

[softwaremaniac]

Confirming Intel card resolved all my issues :)

 

[jgreco]

Confirming Intel card resolved all my issues :)

Very happy to hear it. Regrets about the difficulties; it is unfair that it is so difficult to obtain high quality hardware in your area of the world.

 

[softwaremaniac]

I had one lying around, so it wasn't too bad. I will need a bigger UPS, though. Not just for the machine, but in general.