Slow write speed with 2.5GbE and mirror

[AnakTeka]

Hi, I just built my first ever small home sever, I use Proxmox and created TrueNAS VM. I'm not sure whether this is the expected speed or not, but I can't seem saturate the 2.5GbE for writing. I created NFS share and mount it from my main PC. Then I copied a certain file and only get 150MByte/sec.

Although iperf3 results seems that I should be able to get ~2.5 Gbits/sec (this is a test from main PC to TrueNAS VM).

Code:

➜  ~ iperf3 -c 192.168.18.19 -p 5201
Connecting to host 192.168.18.19, port 5201
[  5] local 192.168.18.233 port 46922 connected to 192.168.18.19 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   268 MBytes  2.24 Gbits/sec    0    428 KBytes       
[  5]   1.00-2.00   sec   271 MBytes  2.27 Gbits/sec    0    455 KBytes       
[  5]   2.00-3.00   sec   279 MBytes  2.34 Gbits/sec    0    477 KBytes       
[  5]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec    0    498 KBytes       
[  5]   4.00-5.00   sec   279 MBytes  2.34 Gbits/sec    0    523 KBytes       
[  5]   5.00-6.00   sec   281 MBytes  2.36 Gbits/sec    0    547 KBytes       
[  5]   6.00-7.00   sec   280 MBytes  2.35 Gbits/sec    0    547 KBytes       
[  5]   7.00-8.00   sec   279 MBytes  2.34 Gbits/sec    0    574 KBytes       
[  5]   8.00-9.00   sec   279 MBytes  2.34 Gbits/sec    0    601 KBytes       
[  5]   9.00-10.00  sec   281 MBytes  2.35 Gbits/sec    0    629 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  2.71 GBytes  2.32 Gbits/sec                  receiver

iperf Done.

Current specs:

• Gigabyte B550I AORUS PRO AX
• Ryzen 4650g
• 1x16GB Kingston ECC Memory on host and shared 8GB to the TrueNAS
• 2x WD40EFRX 4TB on mirror
• PCI passthrough the onboard SATA controller to the VM
• RTL8125 2.5GbE Controller on Proxmox host

Current topology:

I'm still fairly new to the Linux, apologize if there are misleading terms that I use. Happy to be corrected.

 

[somethingweird]

My guess is not enough RAM (8 is barely enough, 16 is better) + RTL8125 isn't helping.

on scale - it only take max 50% of ram for zfs,

 

[chuck32]

What speed do you expect? 150 MB/s isn't awfully slow for s single disk performance, but certainly not on the fast side. Given they are 5400 rpm drives.

You only have a two disk mirror, as far as I know this will not increase write performance, you want striped vdevs for that. In your case the writing of the data is not spread across multiple drives. You will be stuck with the write performance of single drive..

 

[LarsR]

According to the spec sheet of the disk, 150 MB/s is the limit of the drive. So if you want higher speed you'd either need more ram or more vdevs in your pool.

 

[Elliot Dierksen]

NFS defaults to sync writes (on most systems), so an SLOG might help after you add some more RAM.

 

[AnakTeka]

What speed do you expect? 150 MB/s isn't awfully slow for s single disk performance, but certainly not on the fast side. Given they are 5400 rpm drives.

You only have a two disk mirror, as far as I know this will not increase write performance, you want striped vdevs for that. In your case the writing of the data is not spread across multiple drives. You will be stuck with the write performance of single drive..

TBH I don't really know what to expect, this is my first time ever with this kind of stuff but at least from the replies it seems the current speed is expected. Thanks everyone for the replies!