optimize TrueNAS setup

arneboeses

Cadet
Joined
Oct 3, 2020
Messages
9
Hi,

first of all - I just started to use TrueNAS and I love it already. Before, my system was running with a plain FreeBSD. Now would like to ask you for an optimization advice to improve i/o speed.

My system:
  • Mainboard: ASUS PL11C-C/4L + KVM
  • RAM: 2x 8GB ECC
  • CPU: Intel i3-9100F (no graphic)
  • Disks:
    • 2x Samsung NVMe 970 Evo Plus 512GB
    • 2x WD Red Pro 8TB 5400RPM 256MB
  • OS: TrueNAS 12.0-RC1
  • Pools
    • SSD = Jails (Mirror)
    • HDD = Storage (Mirror)
I just thought about to add 2x 250GB Samsung Evo Plus NVMe as VDev (meta or caching) via USB3.1 Gen1 to speed up the HDD pool as the disks are not the fastest but hosts files which are accessed via a web server. Would that make sense?

Looking forward to your thoughts/advises.

Best,
Arne
 

rob90033

Dabbler
Joined
Jan 1, 2015
Messages
20
I am learning myself, been searching for ways to optimize. Did you learn anything?
 

Flashbin

Dabbler
Joined
Jan 29, 2020
Messages
17
So I assume you would put the NVMe SSD's in a USB Disk Enclosure? I don't think that makes sense since USB 3.1 Gen 1 (300 MB/s) is pretty slow compared to NVMe (The Evo Plus reaches 3500 MB/s write, when connected over PCIe of course). Even SATA 3 reaches 600 MB/s. So you would just be slowing everything down. Correct me if I'm wrong.
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
RAM: 2x 8GB ECC

With only that much RAMs, your overall speed will never be that fast...

2x WD Red Pro 8TB 5400RPM 256MB

With only 2 drives, again, your overall speed will never be very fast...

via USB3.1

As mentionned, USB is not that fast and in general, to be avoided with FreeNAS / TrueNAS...

Would that make sense?

Not to me...

First thing would be to go for a pool of at least 8 drives in mirror. To have 4 vDev will be a major boost compared to a single one.

Second thing is about RAM. Anything below 64G is not designed for high performance. It can perform reasonably and be good enough for many but if you are chasing performance, reach that 64G mark.

Third will be to go for actual server gear. Your CPU is designed for desktop usage.

Out of 2 drives / 1 vdev and 16G of RAM, you can not do miracles...
 

Flashbin

Dabbler
Joined
Jan 29, 2020
Messages
17
Anything below 64G is not designed for high performance.
I mean, I've got 16GB total, 8GB are used for my VMs so 8GB for ZFS Cache. Thats more than enough for my use case (heaviest load I've got is streaming video). I easily max out a Gigabit connection. With his setup, 64GB of RAM won't to anything.

Third will be to go for actual server gear. Your CPU is designed for desktop usage.
Again, his CPU is plenty fast for normal stuff. Mine is about the same speed if you can trust userbenchmark.com, and I have multiple VM's running on that too, including a Plex Server with multiple users at a time.

The next step in my opinion would be to move to 10 Gigabit Networking, because with Gigabit even the slowest HDD will max it out.
I could be entirely wrong here, since the exact use case is unknown. But from my experience, you can have very little RAM and a very slow CPU and still get very good performance with TrueNAS.
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
FreeNAS by itself requires a minimum of 8Gig. When running VMs on top of that, you need even more.

and I have multiple VM's running on that too, including a Plex Server

Jails and VMs are not the same... Jails do not need as much RAM as VMs. Still, even for running jails, 16G of RAM is a very minimum if sufficient.
 

Flashbin

Dabbler
Joined
Jan 29, 2020
Messages
17
Jails and VMs are not the same...
Yeah, I know. I have 5 VMs (not Jails), including that Plex VM. 16GB are enough for me, as I said. 8GB for ZFS Cache are plenty for my use case.

I even ran it with 4GB for ZFS Cache for a few days since I was testing something with my VMs. Ran just fine (Plex streaming with multiple users). Again, I can get away with less than 8GB but another use case may need more. 64GB is way too much, except for extreme workloads.
 
Top