First TrueNAS build - Request advice/comments

[LordGardenGnome]

Hello,

Super TrueNAS noob here, but not IT noob.

So I currently have a "server" (running Windows 10), that I've filled up and have decided to do a new build, with TrueNAS (probably Scale). I've already started down the path of parts, I did buy a 45 Drives Q30 with ATX PSU (I looked around for a new chassis for a good 8 months before selecting it). I have an LSI 9305-24i (bought forever ago). I currently only use it for PLEX, I'd like to do more with the new build. I know this is probably a very over ambitious build, that will be overkill for most of what I currently.

Current build:
Asus X299-WS board
Intel i9-7900X
128gb DDR4 2666 unbuffered
RTX 4000 (will be reused in new build)
2x RAID 6 Arrays on an LSI 9211-8i
-4x 10TB WD Red (SMR)
-4x 8TB WD Red (SMR)
An Intel Optane 930 u.2 for caching meta data (might be reused in new build)
Sea sonic 1000kw prime titanium (will be reused in new build)

New build:
Either:
-ASRock Rack EPC621D8A
-SuperMicro X11SPI-TF
Intel Xeon Scalable
Either Silver 4116 or Gold 6126
128gb of ECC Registered or LRDIMM (Depends on how much my total will be)
Some form of RAID 1 SSD for TrueNAS OS
m.2 to u.2 for Intel 903 as a L2ARC (still don't understand SLOG/L2ARC)
15x Seagate EXOS 12TB or 14TB drives (depends one the deals at the time, pricing between 10,12,14TB is close)

Planned to do a RAIDZ2 setup
*******
RAID ZFS Calculator
This is where I get lost-
With a RAIDZ2 pool I was thinking 2 VDEVs - 6 Drives each VDEV. With 1-3 ready spares? Do I need Hot spares, like I would with a RAID 6 array?


I've done lots of reading on VDEV's and pools, I'm trying to setup a simple server at work (just needs a cheap HBA), to play and learn on.

 

[artlessknave]

Some form of RAID 1 SSD for TrueNAS OS

what does "some form of RAID 1 SSD" mean? SSD's aren't RAID.
zfs doesn't have "RAID1", because zfs is not RAID. the ZFS equivalent is mirrors.

m.2 to u.2 for Intel 903 as a L2ARC (still don't understand SLOG/L2ARC)

L2ARC is a read-only copy of ARC on disk (read cache). that's it. memory is always read cache (RAM=ARC) (DISK=level2ARC). ARC is always faster than L2ARC.
SLOG ONLY applies to SYNC writes. SLOG guarantees data is written to actual disk before returning "Datawritten" to the writting application. without a SLOG, all data is stored in ARC for ~ 5 seconds and then flushed to the pool in a contiguous write. SLOG is really only for databases and VM's, with SYNC writes, where a lost write can cause corruption.
it's very unlikely either is worth your time.

With a RAIDZ2 pool I was thinking 2 VDEVs - 6 Drives each VDEV. With 1-3 ready spares? Do I need Hot spares, like I would with a RAID 6 array?

you don't really "need" hotspares with raid or zfs. hotspares are for when you can't get to the server in a reasonable amount of time to replace a drive and you need 100% uptime on a system that is in constant use. a home media server basically never constitues this, instead, a hot spare just wears out the drive faster, since its age is functionally how long it has been "hot"
raidz2 is probably fine, though the drives are pretty large, you might consider 2xraidz3x7, or, depending on chassis (i dont see it listed) 2xraidz3x8
note that, if you plan your topology, you can start out with only 1 raidz and add another later. raidz2/3 of 6-7 12TB disks is a fair bit of space

2x RAID 6 Arrays on an LSI 9211-8i

are you running ZFS ON top of RAID, on SMR disks?
*cries*

 

[LordGardenGnome]

what does "some form of RAID 1 SSD" mean? SSD's aren't RAID.
zfs doesn't have "RAID1", because zfs is not RAID. the ZFS equivalent is mirrors.

Apologies, A ZFS mirror for the OS. Utilizing two SSD's

raidz2 is probably fine, though the drives are pretty large, you might consider 2xraidz3x7
note that, if you plan your topology, you can start out with only 1 raidz and add another later. raidz2/3 of 6-7 12TB disks is a fair bit of space

Cool, that's something I was trying to understand. With my current stuff, I have 42TB used between the (current media server internal and external space). As the Hot spares, it's just something I did in the past, but thanks for that info.

are you running ZFS ON top of RAID, on SMR disks?
*cries*

oh no, it's a LSI running it's regular Mega-RAID with two volumes. Each array is just that, no ZFS running.

L2ARC is a read-only copy of ARC on disk (read cache). that's it. memory is always read cache (ARC) (level2 ARC). ARC is always faster than L2ARC.
SLOG ONLY applies to SYNC writes. SLOG guarantees data is written to actual disk before returning "Datawritten" to the writting application. without a SLOG, all data is stored in ARC for ~ 5 seconds and then flushed to the pool in a contiguous write. SLOG is really only for databases and VM's, with SYNC writes, where a lost write can cause corruption.
it's very unlikely either is worth your time.

Cool, thanks for that.

I've never had anyone locally to me that's tried any of this, or I haven't found anyone, so it's just been lots of reading and youtube, and some friends when they have time to explain.

 

[artlessknave]

my sig has a number of links, intended to help the new.

oh no, it's a LSI running it's regular Mega-RAID with two volumes. Each array is just that, no ZFS running.

oops, you said it was windows, my bad. read failure.

-SuperMicro X11SPI-TF

I'm basically drooling looking at this. I want something similar but $2000 to replace my working NAS right now is a bit much.
I wanted to do Epyc but aparently it's low end power usage is stupid, like 180W idle, which is way too much.

supermicro is extremely popular here, with asrock generally being a black sheep. those in the truenas forums often are very "Data integrity" oriented and see many people trying to run it on an AMD APU from 20 years ago...and so often greatly encourage known good working hardware.
even your old system is fairly decent; while it's not servergade, it's not "gaming" grade either.