Save energy and boost peformance with NVME SSD ???

[Stephan0711]

Dear All,

I am planing to upgrade my home server with a NVME SDD (2TB SN700 WD RED NAS SSD)
I have several questions because of the change…

My home server:
Xeon 1230L v5
16GBx4=64GB total DDR4 ECC RAM
Intel Intel S1200SPLR mainboard
10GBit Mellanox X3

6 HD WD RED 4TB in Raid-Z2 =16TB pool for my data and jails, until now also used for downloads
Jails = jdownloader, torrent, maybe Pihole/DNS server…

Boot device only fort he OS mSATA SSD 64GB Transcend

Now on Truenas Core, but I will switch to Scale with this build...


Background:
I already ordered a NVME M.2 SSD SN700 WD NAS SSD with 2TB
My plan is to save energy by using this ssd for downloads. Most downloads are streamed to my TV and will be deleted , some data I want to keep will be copied to my hdd pool.

Questions about saving energy:

1. I have some jails on the hdd pool, is this a problem because it will prevent going the disks to go to sleep mode? I am thinking on jdownloader or a DNS server?
2. Can I manually set the time until the disks fall to sleep? What parameters should I change, what would you recommend?

I also want to speed up my hdd pool with caching and also add a boot mirror for safety:

Questions:

1. I think 64GB is overkill for a boot drive with truenas scale? To save space on my second nvme ssd (mirror…) I am planing to partion the boot ssd from 64 to 40GB, I that enough? What size do you recommend?
2. ZFS intend log with 6GB (10G network) would be nice, but I think this will kill the new NVME SSD to fast?
3. I am planning to use a L2ARC cache form my hdd pool. I assume with 64Gb ram the L2ARC should be 3 or 5 times the ram. I am planing to start with 256GB. Is this okay?I want to speed up the hdd pool, especially my jails. Will the L2ARC help with the jails stored on the hdd pool?

1. I also had the idea to add a ZFS metadata cache (special device). Obviosly this should be mirrored. My idea was to use the other half of my SATA SSD for it and mirror it with a partion on the new nvme ssd... Unfortunatelly I found a rule of thumb that it should have 0,3% of the hdd pool. (=16.384GB x 0,003 = 491GB) So my 64TB msata SSD would be far to small?

Thank you for your help and sorry form my english, I am not a native speaker…
Truenas and this community is awsome !!!!!!!!
Regrads Stephan

 

[artlessknave]

I have some jails on the hdd pool, is this a problem because it will prevent going the disks to go to sleep mode? I am thinking on jdownloader or a DNS server?

it will not cause a problem but as long as the jail is running the HDD pool will never spin down. move your jails to the ssd.

Can I manually set the time until the disks fall to sleep? What parameters should I change, what would you recommend?

yes.
however, there are mixed reports and this if often a contentious topic. its generally considered a waste of time trying to spin disks up and down, as that has a good chance of putting enough stress on the disks to cause sufficient failures of drives to offset any possible power savings. these typically cost maybe a few dollars *a year* to run 24/7, so any failure will wipe that out. the system itself will use enough power to make the drives idle power usage irelevant, plus spin up power can stress your power supply and the rest of the system.

ZFS intend log with 6GB (10G network) would be nice, but I think this will kill the new NVME SSD to fast?

this will kill anything but the highest endurance SSDs very fast for no benefit in vast majority of home use cases. at all.
SLOG (this is what you are referring to, as the ZIL ONLY exists in pool) is not a write cache, will only apply to sync writes, and SMB cannot interact with sync writing functionality. at all. the ZFS write cache is always in RAM; a SLOG provides a backup of the write cache for sync writes, and will never be read from in normal operations, 100% writes. SLOG+sync writes are almost always slower than async writes as zfs has to write the data 2 times. SLOG is for data you absolutely cannot have missing; databases and VM's primarily, where a single lost write can corrupt the whole VM/DB. home users aren't going to see this generally. if your NAS crashes while you are writing a video to it, you can just restart copying it when its stable again.

I think 64GB is overkill for a boot drive with truenas scale? To save space on my second nvme ssd (mirror…) I am planing to partion the boot ssd from 64 to 40GB, I that enough? What size do you recommend?

this is another contentious topic, however, you cannot partition the boot disks with truenas. do not even try, this is unsupported and is known to causes issues. boot disks are dedicated disks. the "waste" will simply improve the life time of the drive due to wear levelling. ideal sizes for boot disks are 30-120GB. I use 120GB disks because finding smaller is more effort than its worth. they are like 20-30$ each and will likely outlive me.

I am planning to use a L2ARC cache form my hdd pool. I assume with 64Gb ram the L2ARC should be 3 or 5 times the ram. I am planing to start with 256GB. Is this okay?I want to speed up the hdd pool, especially my jails. Will the L2ARC help with the jails stored on the hdd pool?

you do not have enough RAM to even consider L2ARC. 128GB min. ARC is always better than L2ARC. you can "speed up" jails by simply moving the jail pool to ssd

I also had the idea to add a ZFS metadata cache (special device). Obviosly this should be mirrored. My idea was to use the other half of my SATA SSD for it and mirror it with a partion on the new nvme ssd... Unfortunatelly I found a rule of thumb that it should have 0,3% of the hdd pool. (=16.384GB x 0,003 = 491GB) So my 64TB msata SSD would be far to small?

no. truenas assumes the boot disks are NOT data disks, and considers them expendable. you have been warned.
there is no "metatdata cache" special vdev provisioned by truenas. the special vdev for metadata becomes the primary storage for that metadata and thus a required vdev for the pool to function. there IS a way to have an l2arc dedicated to metadata, which would be a metadata cache, however, this is outside of what truenas provides and isnt really that useful. ARC already deals with this.


there are other NAS solutions that allow much of what you are proposing but truenas is desgined as an appliance with every truenas install having a predictable topology that ensure data and OS disks never get mixed.