TrueNas 12 - Slow - Onto Permissions :/

[Bobsnet]

Hi Im hoping someone can help me with slow write speed issues. Ive read a lot but not sure what is relevant.

Dell T620 - ESXI 6.7
HBA Passthrough to TrueNas VM with 4 vCPU and 32GB ram
8 WD RED 3TB
VMXNET3 single NIC

ESXI is installed on a SD, and the datastore is a separate set of RAID SSD's

When I run a speed test using iperf between truenas (Server side) and a windows VM (client side) i get the following, not great but useable speeds

Code:

C:\>iperf3 -c 192.168.200.10
Connecting to host 192.168.200.10, port 5201
[  4] local 192.168.200.254 port 50133 connected to 192.168.200.10 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   278 MBytes  2.33 Gbits/sec
[  4]   1.00-2.00   sec   291 MBytes  2.44 Gbits/sec
[  4]   2.00-3.00   sec   242 MBytes  2.03 Gbits/sec
[  4]   3.00-4.00   sec   240 MBytes  2.01 Gbits/sec
[  4]   4.00-5.00   sec   237 MBytes  1.99 Gbits/sec
[  4]   5.00-6.00   sec   234 MBytes  1.96 Gbits/sec
[  4]   6.00-7.01   sec   135 MBytes  1.12 Gbits/sec
[  4]   7.01-8.00   sec   209 MBytes  1.76 Gbits/sec
[  4]   8.00-9.00   sec   168 MBytes  1.41 Gbits/sec
[  4]   9.00-10.00  sec   257 MBytes  2.16 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  2.24 GBytes  1.92 Gbits/sec                  sender
[  4]   0.00-10.00  sec  2.24 GBytes  1.92 Gbits/sec                  receiver

When running the same from either another ubuntu VM, or my desktop windows PC its a much different output

Code:

D:\>iperf3 -c 192.168.200.10
Connecting to host 192.168.200.10, port 5201
[  4] local 192.168.128.182 port 52582 connected to 192.168.200.10 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  3.00 MBytes  25.1 Mbits/sec
[  4]   1.00-2.01   sec  3.12 MBytes  26.1 Mbits/sec
[  4]   2.01-3.00   sec  3.12 MBytes  26.3 Mbits/sec
[  4]   3.00-4.01   sec  3.25 MBytes  27.0 Mbits/sec
[  4]   4.01-5.01   sec  3.12 MBytes  26.1 Mbits/sec
[  4]   5.01-6.01   sec  3.25 MBytes  27.5 Mbits/sec
[  4]   6.01-7.01   sec  3.25 MBytes  27.1 Mbits/sec
[  4]   7.01-8.00   sec  3.38 MBytes  28.5 Mbits/sec
[  4]   8.00-9.01   sec  3.38 MBytes  28.0 Mbits/sec
[  4]   9.01-10.01  sec  3.25 MBytes  27.5 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.01  sec  32.1 MBytes  26.9 Mbits/sec                  sender
[  4]   0.00-10.01  sec  32.1 MBytes  26.9 Mbits/sec                  receiver

iperf Done.

I have also tested a file transfer by both FTP and SMB from windows machine and get similar speeds (Large files 4gb+)
Currently Vswitch has 1 uplink, through a PCIE 4 port ethernet card (INTEL) and only the truenas and windows VM's are on this switch
The vswitch is configured for Jumbo frames with an MTU of 9000, connected to a meraki switch with MTU (Standard) 9578 (MS220-8P) I am unable to find an MTU configuration for my MX firewall, and have noticed that when I traceroute from my main PC to TrueNAS, one of the hops is the MX, even though both server and PC are connected to the same switch, so Im thinking it could possibly be a routing issue.

For added clarity, shares are setup as both SMB and NFS (need to be accessed by both win and linux)

Anyone any pointers? Apologies if i have missed out any required info, I have tried to be as complete as possible.

 

[Bobsnet]

Further to this, When I access truenas from my main PC (on a different VLAN) the GUI does not ever load fully, but from my windows VM (on the same VLAN as truenas) it does. I am starting to think more and more this is a network configuration issue.

 

[Bobsnet]

I have narrowed this down to be a network issue. When I move my main PC to the same VLAN as the TrueNAS I get transfer speeds of around 900Mbit, all be it in SMB is this around 100Mbit. Im wondering could it possibly be the MTU on the Meraki Firewall. It can not be configured (as per meraki support) so Im not sure what the workaround is for me here

 

[Bobsnet]

Im a bit further along, but now have some issue with permissions. I have Sonarr Radarr etc running on a separate VM, with NFS mount shares via fstab between the two. When Sonarr goes to transfer a file from the NZB folder I can see the file start to transfer, but as soon as complete the file disappears and the transfer fails. I can see that the owner of the new file is different from the rest of the pre existing files, which is to be expected as they were imported from backup, this is for newly downloaded files.

Just to add, NZBGET is running in another container on the same docker machine, and can write the downloaded file to a share also on the same truenas server.

I have read about linux permissions till my eyes hurt, but just dont get it. Do the UUID and GUIDs need to match across all linux machines? and should these then be the UID and GID that are transported into docker (since services in docker run as abc?)

Thanks in advance!

 

[Bobsnet]

Final add - Transmission, also running on a container in the same machine and sonarr can collect and move the files into the same share that NZBget is failing to move too

 

[Bobsnet]

So, Ive no worked out that if I chmod777 the folder and file that the downloaded content is in from the ubuntu hosting the nzbget container then the file is moved correctly, however, these permissions dont get added when a new download is grabbed