Making some changes to my TrueNAS setup - validate my approach?

[hungarianhc]

Hey there - I just moved all of my equipment into a new rack. One bummer is that I mis-planned my rack units. I literally made a spreadsheet to manage my 12U rack... but then I put in 13 rows. Ugh. The net result is that I've moved from a 3U NAS chassis to a 2U chassis. That's not super relevant, I suppose, until later. I run TrueNAS on bare metal, and I run some VMs / jails, including HomeAssistant, Plex, Syncthing, and others. I'm starting to do more video editing, and I would like to be able to edit video directly off the NAS.

Current setup:
5 x 6TB drives in RAIDZ2, 18TB usable
AMD Ryzen 5 5600X CPU
128GB ECC RAM
10G ethernet connection to Ubiquiti UDM Pro

Optimizing for video editing:
I'd like to edit video directly off the NAS, and from the research I've done, and the fact that most of my 128GB of RAM is used as ZFS cache, a ZIL won't do much, and I can't imagine L2ARC will do much either. Is that fair? It just feels like those won't help for video editing use-case at all. Rather, my motherboard does have TWO PCIe4.0 m.2 slots, and I'm considering putting dual fast SSDs in there, striped. This won't be for any permanent data or anything like that. Once I create a video, it gets archived and placed on the current pool I have. Would you recommend an approach other than a "working" pool that is two striped SSDs? Let me know if I'm missing some type of caching option for my primary pool or if you'd recommend a different approach.

Reducing my drives:
This one is painful, but when I surprisingly had to drop from a 3U to 2U case, my ability to have a bunch of drives went down. Right now I do have 5 drives jammed into my case, but for the thermal health of my system, I'm thinking of dropping down to 3. Is there any concern with switching my RAIDZ2 setup to 3 drives? I still want dual drive parity. I'm thinking of just paying the premium for 18TB drives, and going 3 x 18TB. Yes... I know... The fewer drives I have, the higher likelihood I have more drives fail while one is resilvering or something like that, but my data backs up off-site with sycthing, and dual drive redundancy makes me feel pretty good, but... other thoughts here?

Thanks in advance for your feedback!

 

[sretalla]

most of my 128GB of RAM is used as ZFS cache, a ZIL won't do much, and I can't imagine L2ARC will do much either. Is that fair?

Depending on the types of writes you're doing (if they are all async, then no benefit from SLOG).

L2ARC would only help if you're seeing big cache miss ratios (or your "working set" is bigger than 100GB or so). (arc_summary at CLI)

Is there any concern with switching my RAIDZ2 setup to 3 drives?

Yes, you can't have RAIDZ2 with only 3 drives... 4 minimum.

 

[Ericloewe]

Yes, you can't have RAIDZ2 with only 3 drives... 4 minimum.

The funny thing is that you can, technically speaking, but why would you?

 

[danb35]

The funny thing is that you can,

Are you sure? Because I don't think you can, technically speaking or otherwise, any more than you could create a two-disk RAIDZ1. I guess I could test that on a VM if I were really curious...

 

[Ericloewe]

Are you sure? Because I don't think you can, technically speaking or otherwise, any more than you could create a two-disk RAIDZ1. I guess I could test that on a VM if I were really curious...

The requirement is P+1, strictly speaking (I think TrueNAS' GUI enforces P+2, which only makes sense).
It's one of those obscure things like (let's see if I get this one right) zfs property=value path/to/dataset being effectively an alias for zfs set property=value path/to/dataset.

 

[sretalla]

Well, SCALE lets me do it from CLI with 3 sparsefiles, so there's at least a technical "yes" to can you do it.

zpool create rz2test raidz2 /root/sparse1 /root/sparse2 /root/sparse3

Code:

scale1# zpool status -v rz2test
  pool: rz2test
 state: ONLINE
config:

        NAME               STATE     READ WRITE CKSUM
        rz2test            ONLINE       0     0     0
          raidz2-0         ONLINE       0     0     0
            /root/sparse1  ONLINE       0     0     0
            /root/sparse2  ONLINE       0     0     0
            /root/sparse3  ONLINE       0     0     0

errors: No known data errors

I don't have a test rig ready with 3 physical (or virtual) disks right now.

 

[hungarianhc]

The funny thing is that you can, technically speaking, but why would you?

I think I gave my use-case... I can only do 3 disks, due to physical capacity issues, but I want 2-disk redundancy. What's the better option? A 3-drive mirror, perhaps?

 

[danb35]

What's the better option? A 3-drive mirror, perhaps?

Yes, exactly.