Slow write speeds to and from my PC. after 5 seconds of data transfer

[bruno.hirt]

Hello everyone!
I'm new to the world of TrueNAS, made it just to store my home files in just one place!

My NAS specs:
AMD Ryzen 3200G
16Gb RAM
4x 3TB Seagate Ironwolf NAS ST3000VN007
1 pool in RaidZ1
1x Nvme 128Gb for TrueNAS OS (I thought I could use a partition of it as SLOG/ZIL, but I discovered too late that I can't)
Intel NIC Expi9400ptblk (the mb realtek one didn't do the job)

My PC specs
Core i7 8700k
32Gb RAM
NVMe storage
Intel NIC
etc

So, as you can see, it's a humble NAS rig, I don't need a lot.

My doubt is that, I was transferring all my stuff to it, and the transfer speeds would start at 140-180MB/s, and that's fine by me, but then, after 5-10 seconds, it would slow downs to a constant 50-65MB/s. If I paused the transfer for 5-10 seconds, and then unpaused it, again it would peak ate 140-180MB/s for some seconds and slow down again.

I have a modem that have a CAT6 cable connected directly into the NAS, and another CAT6 cable connected directly to my PC.
Iperf tests:
NAS->PC: 300-360MB/s, very stable speeds (seems ok, 3 times the read speed of a single drive as RaidZ1 predicted)
PC->NAS: 120-350MB/s, would change abruptly from 300MB/s to 120MB/s and up to 230MB/s again, over and over, but it averaged 220MB/s in a 2 minute test

Anyway, faster than what I'm having when transferring files from my PC to my NAS (NVMe storage over CAT6 cable to modem and them to NAS [4x 3Tb RaidZ1]) - 60MB/s

Is it something wrong? Maybe I need to add cache storage for it to run smoother??
Maybe I could run the OS from a USB stick and use my NVMe 128Gb as cache?

Thank you very much!
Have a nice day!

 

[ChrisRJ]

ZFS does not have a write cache in the classical sense. Please check out https://www.truenas.com/community/threads/some-insights-into-slog-zil-with-zfs-on-freenas.13633/ for details.

 

[c77dk]

The 5 seconds is probably the txg being full - which will say it's directly to ram, and when it drops you're at what the pool can sustain.

Those numbers with iperf - something's wrong. With 1Gbps connection you can't get that throughput (theoretical you can get 125MB/s, and that's including all overhead) - and if it should be Mbps then they're quite low.

 

[bruno.hirt]

That's what I thought about the iperf results, because IRL, with several small files being copied to the NAS it just achieves 100KB/s (most of the time around 20-30KB/s). Big single files, like a movie, achieves the 60MB/s I told you about.

And what's interesting as well is that my Read speeds are not 3x the speed of a single driver, as I'm getting 60MB/s as well on Read speeds when copying a big file from the NAS to my PC, and that's definitely not a limitation of my PC NVMe.

I can't understand it, to be honest

 

[morganL]

There are several key parameters when looking at performance:

Network speed
I/O size
Record/block size
Queue depth... how many outstanding I/O requests are there?