New TrueNAS Scale build - would appreciate critique/advice

[Friikazoid]

Hello everyone!

I've been working on getting better educated on TrueNAS SCALE and suitable hardware and have finally arrived at a plan. Before I pull the trigger, I wanted to get some community feedback, critique and advice on that plan. If you read this post and see something glaring out that you that I may have missed, I'll really appreciate if you let me know!

Thank you!!

Purpose of build:

• Starting out, the machine will serve as a Plex server and as a family file-server
• As I become more familiar with the software available / virtualization / etc. I will likely grow the use cases
• The goal of my build is to use hardware, which allows me to expand use cases as I learn, with simple hardware addons (add-in card, SAS controller, extra RAM, etc.)

Proposed build:

Motherboard: SUPERMICRO MBD-X12SPA-TF-O Extended ATX Server Motherboard LGA 4189 Intel C621A

• Ice Lake server socket (to let me use either Xeon Silver brand new, or some platinum CPUs used from eBay)
• 16 DDR4 RAM slots --> plenty of space to increase RAM over time as needs grow for ARC. DDR4 RAM as DDR5 is still super expensive and hard to get
• 4 M.2 PCIe 4.0 x4 slots --> SSD for boot + 2 slots for Optane SLOG (mirrored) + 1 spare for future needs (L2ARC)
• 64L of PCIe - plenty for a dual slot GPU + networking card + SAS controllers (when they become necessary)
• 8 SATA3 slots for starting out with storage without immediate need for SAS controller

CPU:
Option 1: Used Ice Lake Xeon Platinum 8352Y 32C/64T

• PRO:
  • Super high core count for any future VM needs
  • Latest available generation (Sapphire will be stupid expensive for a while)
• CON:
  • Risk of used CPU (though I know Xeons are built to last)
  • Lower frequency (2.2GHz base, 3.4GHz turbo, does have speed step with 24 and 16 cores active)

Option 2: New Ice Lake Xeon Silver 4314 16C/32T

• PRO:
  • Higher base frequency, lower power at max frequency
  • New
• CON:
  • Half-core count - lower potential for VMs, but 16C is still plenty

CPU cooler: Noctua NH-US14 DX-4189

I am leaning towards the peace of mind of option 2 (also 350 bucks cheaper)

RAM: 128GB (2x64GB) DDR4-3200 ECC LRDIMM

• Highest frequency RAM Ice Lake can handle. With16 slots, could go up to 1TB of RAM
• Was thinking about a single 128GB stick, but would rather have 2 sticks starting out in case one fails.
• If I start needing to grow faster, I'll swap to 128GB sticks

M.2:
Boot SSD: Will find a decent 256GB SSD
SLOG: 2x Intel Optane 905P U.2 (with converter to M.2) mirrored for backup
+1 spare for later on (L2ARC)

Graphics Card: RTX3050 8GB

• Initially used mainly for Plex hardware transcode. I know it's limited to 3 simultaneous transcodes (should be sufficient starting out)
• Will replace with a workstation graphics cards with unlimited transcodes when necessary (thinking an RTX A2000 12GB)

HDD: 6x Seagate IronWold PRO 18TB NAS HDD in RaidZ2

• All 6 drives in single vdev RaidZ2
• If I need to expand the zpool in the future, I'll always add a new vdev of 6 drives (18TB) in RaidZ2

(In the future) SAS controller: LSI 9300-16i 12Gbps PCIe 3 8x

• Flashed by theartofserver (seller) to IT mode
• Will provide all the connections needed that could fit into my case

Case: Fractal Design Define 7 XL

• Space for up to 18 3.5" HDDs (so should be good up to 3 vdevs of 6HDDs RaidZ2 each in pool)
• Supports E-ATX
• I was looking at used JBOD chassis (like the this used Supermicro) but it seems like i) it's too overkill at this point ii) only allows for half-size expansion cards (so makes GPUs / etc. more challenging to find) and iii) don't really have any space to put it at this time
• Eventually I could migrate to a JBOD chassis when my need grows for it

Power Supply: EVGA SuperNOVA 1000 P6 1000W PSU


Like I said - I'll appreciate any thoughts and again THANK YOU!!!

 

[danb35]

SLOG: 2x Intel Optane 905P U.2 (with converter to M.2) mirrored for backup

Unnecessary for your stated use case.

 

[ChrisRJ]

HDD: 6x Seagate IronWold PRO 18TB NAS HDD in RaidZ2

• All 6 drives in single vdev RaidZ2
• If I need to expand the zpool in the future, I'll always add a new vdev of 6 drives (18TB) in RaidZ2

I would probably go for 2* RAIDZ2 with 8 drives each. This will give more or less the same net capacity but save 2 drives. Yes, you get fewer IOPS, but if they are a concern the 3* RAIDZ2 is also far from ideal.

The Seagate Exos drives are often cheaper than then Iron Wolf (Pro), so I would check them out as well.

Case: Fractal Design Define 7 XL

• Space for up to 18 3.5" HDDs (so should be good up to 3 vdevs of 6HDDs RaidZ2 each in pool)

You need to be careful with cooling for such a number of drives. They will be densely packed, so fans with high pressure are in order. Those tend to be (very) loud.

 

[Friikazoid]

Unnecessary for your stated use case

I figured :). I threw those in just because I can get them ridiculously cheap through my company and figured "they might not be around for much longer" based on the decisions Intel is making.

 

[Friikazoid]

I would probably go for 2* RAIDZ2 with 8 drives each. This will give more or less the same net capacity but save 2 drives. Yes, you get fewer IOPS, but if they are a concern the 3* RAIDZ2 is also far from ideal.

I think IOPS in the early stages shouldn't be a key concern. If I may ask - what would you recommend as a solution for IOPS in terms of vdevs configs? I was just thinking I'd go with 6, then add 6 for smaller incremental chunks , then add 6 more and probably decide to get a new case/chassis at that point anyway if I know myself :)

The Seagate Exos drives are often cheaper than then Iron Wolf (Pro), so I would check them out as well.

Great, I'll check them out!

You need to be careful with cooling for such a number of drives. They will be densely packed, so fans with high pressure are in order. Those tend to be (very) loud.

Good point! I think by the time I'd be going for the jump from 12 to 18 (or I'll see now if it will be from 16 to 24 if I do the 8 drive configuration) I'll start looking for a used chase with 24+ bays and a little more ideal airflow.

Unless ... do you have any recommendations? Everything I seem to find is 36 bays with 12 in the back, leading to half-height expansion cards needed.

 

[Etorix]

Hoarding Optane drives "while supplies last" is indeed tempting. But it remains that you have no use case for it yet. And your build is massive overkill for a home NAS—starting with the 3rd gen. Xeon Scalable and the expensive workstation motherboard.

 

[ChrisRJ]

what would you recommend as a solution for IOPS in terms of vdevs configs?

Mirrors.

But without at least somewhat specific requirements it is a bit of a philosophical discussion. You can always go faster, bigger, etc. The questions is whether you need it and what you are willing to pay.