Best configuration optimized for writing speed

[spideraxal]

~Before reading, known that I do not need an exact answer because real life performance isn't easy to aproximate, just the best tips on how to make the most of my setup.~

Hello everybody,

I currently have TrueNas running on older hardware (i7 970 - Asus P6T WS pro & 28GBs of RAM) with 2 pools: One with older 2x WD Re drives in Stripe (don't care about the data) and the other with 4x WD Red Pro 4TB drives in RaidZ1 (single vdev). I bought an external HBA Card (LSI SAS2008), because I only have 6 internal Satas, together with a 10Gb Asus XG-C100C Card.

The purpose of the NAS is to store large video footage (currently have ~10TB of video material). Because of camera upgrades (HD -> 4K), I am in need of a writing speed increase, for faster ingestion.

My write speeds are somewhere between 290-300 MB/s when copying stuff to the NAS using Windows SMB. I already bought 3x WD Red Pro 16TB drives (they were on sale) and will buy some better CPU+Mobo combo within the next month.

I've been using this setup for ~2 years now, but recently started digging more into ZFS and found out you can attach multiple vdevs to a pool and if the vdevs are distributed correctly, based on their seizes, you can get a performance improvement for your write speed (based on benchmarks I saw posted by users on this forum).

My question is now, what would be the best way to set up my pools and vdevs, for maximum performance?

Thank you in advance.

 

[ChrisRJ]

What kind of write are we looking at? Sequential or random? File size? Can you be more specific about your use-case?

 

[NugentS]

Also Scale or Core. That Asus is an Aquantia chipset (I think) which I thought was not well supported in Core

 

[sretalla]

when copying stuff to the NAS using Windows SMB

store large video footage

If you're a video editing guy, maybe you're also using a Mac to upload from... which I think in some cases can ask for Sync writes from SMB shares.

The first thing to test with that would be to set up a dataset (and a new share for it) with sync=disabled and see if that improves performance (confirming that sync writes are indeed requested). Also consider setting the recordsize for your media datasets to a much bigger number like 1MB.

If that's the case, you can either decide to be satisfied that you will always verify the file once it arrives and keep the copy on the client until having done so, which would eliminate any real need for the sync writes... then enjoy your increased performance more-or-less for free.

If it doesn't change anything...

And you're only copying large media files (big sequential writing... which would already be confirmed as asynchronous at this point)...

You may benefit from messing around with more VDEVs (mirrors) or different widths of VDEV in RAIDZ (1 or 2)... we really should be advising you to get away from RAIDZ1 unless you're really comfortable that your backups are sufficiently frequent and trustworthy (resilvering disk as large as you're discussing will take days and the stress on the remaining disks is significant while you have no redundancy).

(I have seen a lot of testing that shows that RAIDZ2 can outperform mirrors for large sequential writes, but numbers of drives in the VDEV matter and don't go linearly up as you add more... can even get worse, then better as you add, etc.)