I'm getting terrible network performance within FreeNAS 12.0 particularly between clients and either FreeNAS jails or VMs. I'm not finding a problem when connecting to the main FreeNAS installation - only the VMs or jails. jails are utilizing the vnet implementation. In terms of hardware I have a SuperMicro X11SSL-CF motherboard. Something tells me its not hardware related since I my iperf3 tests (shown below) between a wireless client located on the same LAN are roughly equivalent between the main freenas installation and another VM not running on FreeNAS. The network speed drops precipitously with jails or VMs implemented by FreeNAS. For what its worth I have all traffic tagged entering the FreeNAS and a serious of bridges associated with each tagged network. Perhaps the introduction of tags is slowing performance. In my tests below the FreeNAS jail and VM were all tied to the same bridge.
Igb0---->>> VLAN1 ---->>> Bridge0 ---->>> Jails and VM (in this test tied to Bridge == vnet0:bridge0)
|
| ---> Main FreeNas Installation
---->>> VLAN20 ---->>> Bridge20
---->>> VLAN30 ---->>> Bridge30
---->>> VLAN40 ---->>> Bridge40
Additional Screenshots showing implementation
iperf 3 results from client to main freenas installation:
iperf3 from client to FreeNAS Arch Linux VM
iperf3 from client to FreeNAS jail with base jail of
iperf3 from client to another LAN VM running within xcp-ng hypervisor -- not associated with FreeNAS
Here are my system tunable network related settings:
net.inet.tcp.recvbuf_max
16777216
SYSCTL
Generated by autotune
net.inet.tcp.recvbuf_inc
524288
SYSCTL
Generated by autotune
net.inet.tcp.reass.maxqueuelen
4096
SYSCTL
net.inet.tcp.mssdflt
1448
SYSCTL
net.inet.tcp.delayed_ack
0
SYSCTL
net.add_addr_allfibs
0
SYSCTL
Igb0---->>> VLAN1 ---->>> Bridge0 ---->>> Jails and VM (in this test tied to Bridge == vnet0:bridge0)
|
| ---> Main FreeNas Installation
---->>> VLAN20 ---->>> Bridge20
---->>> VLAN30 ---->>> Bridge30
---->>> VLAN40 ---->>> Bridge40
Additional Screenshots showing implementation
iperf 3 results from client to main freenas installation:
Code:
iperf3 -c 10.0.1.197 -f K Connecting to host 10.0.1.197, port 5201 [ 5] local 10.0.1.184 port 56885 connected to 10.0.1.197 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 3.19 MBytes 3263 KBytes/sec [ 5] 1.00-2.00 sec 5.97 MBytes 6105 KBytes/sec [ 5] 2.00-3.00 sec 6.73 MBytes 6901 KBytes/sec [ 5] 3.00-4.00 sec 6.27 MBytes 6398 KBytes/sec [ 5] 4.00-5.00 sec 7.11 MBytes 7303 KBytes/sec [ 5] 5.00-6.00 sec 7.42 MBytes 7600 KBytes/sec [ 5] 6.00-7.00 sec 6.87 MBytes 7005 KBytes/sec [ 5] 7.00-8.00 sec 7.23 MBytes 7431 KBytes/sec [ 5] 8.00-9.00 sec 6.51 MBytes 6661 KBytes/sec [ 5] 9.00-10.00 sec 7.18 MBytes 7347 KBytes/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 64.5 MBytes 6601 KBytes/sec sender [ 5] 0.00-10.04 sec 64.4 MBytes 6571 KBytes/sec receiver iperf Done.
iperf3 from client to FreeNAS Arch Linux VM
Code:
iperf3 -c 10.0.1.86 -f K Connecting to host 10.0.1.86, port 5201 [ 5] local 10.0.1.184 port 53855 connected to 10.0.1.86 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 203 KBytes 202 KBytes/sec [ 5] 1.00-2.00 sec 45.2 KBytes 45.2 KBytes/sec [ 5] 2.00-3.00 sec 18.4 KBytes 18.5 KBytes/sec [ 5] 3.00-4.00 sec 12.7 KBytes 12.7 KBytes/sec [ 5] 4.00-5.00 sec 24.0 KBytes 24.0 KBytes/sec [ 5] 5.00-6.00 sec 58.0 KBytes 58.2 KBytes/sec [ 5] 6.00-7.00 sec 43.8 KBytes 43.7 KBytes/sec [ 5] 7.00-8.00 sec 43.8 KBytes 43.8 KBytes/sec [ 5] 8.00-9.00 sec 66.5 KBytes 66.5 KBytes/sec [ 5] 9.00-10.00 sec 38.2 KBytes 38.2 KBytes/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 554 KBytes 55.3 KBytes/sec sender [ 5] 0.00-10.02 sec 427 KBytes 42.6 KBytes/sec receiver iperf Done
iperf3 from client to FreeNAS jail with base jail of
Code:
iperf3 -c 10.0.1.158 -f K Connecting to host 10.0.1.158, port 5201 [ 5] local 10.0.1.184 port 54231 connected to 10.0.1.158 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 158 KBytes 157 KBytes/sec [ 5] 1.00-2.00 sec 67.9 KBytes 68.1 KBytes/sec [ 5] 2.00-3.00 sec 26.9 KBytes 26.8 KBytes/sec [ 5] 3.00-4.00 sec 9.90 KBytes 9.94 KBytes/sec [ 5] 4.00-5.00 sec 42.4 KBytes 42.4 KBytes/sec [ 5] 5.00-6.00 sec 43.8 KBytes 43.9 KBytes/sec [ 5] 6.00-7.00 sec 65.0 KBytes 64.9 KBytes/sec [ 5] 7.00-8.00 sec 43.8 KBytes 43.9 KBytes/sec [ 5] 8.00-9.00 sec 55.1 KBytes 55.2 KBytes/sec [ 5] 9.00-10.00 sec 26.9 KBytes 26.9 KBytes/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 539 KBytes 53.9 KBytes/sec sender [ 5] 0.00-10.02 sec 413 KBytes 41.2 KBytes/sec receiver iperf Done.
iperf3 from client to another LAN VM running within xcp-ng hypervisor -- not associated with FreeNAS
Code:
perf3 -c 10.0.1.11 -f K Connecting to host 10.0.1.11, port 5201 [ 5] local 10.0.1.184 port 56503 connected to 10.0.1.11 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 6.15 MBytes 6291 KBytes/sec [ 5] 1.00-2.00 sec 6.66 MBytes 6826 KBytes/sec [ 5] 2.00-3.00 sec 4.26 MBytes 4354 KBytes/sec [ 5] 3.00-4.00 sec 7.38 MBytes 7567 KBytes/sec [ 5] 4.00-5.00 sec 8.76 MBytes 8964 KBytes/sec [ 5] 5.00-6.00 sec 8.07 MBytes 8254 KBytes/sec [ 5] 6.00-7.00 sec 7.54 MBytes 7710 KBytes/sec [ 5] 7.00-8.00 sec 6.02 MBytes 6181 KBytes/sec [ 5] 8.00-9.00 sec 7.26 MBytes 7431 KBytes/sec [ 5] 9.00-10.00 sec 7.68 MBytes 7852 KBytes/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.00 sec 69.8 MBytes 7143 KBytes/sec sender [ 5] 0.00-10.01 sec 69.2 MBytes 7082 KBytes/sec receiver iperf Done.
Here are my system tunable network related settings:
net.inet.tcp.recvbuf_max
16777216
SYSCTL
Generated by autotune
net.inet.tcp.recvbuf_inc
524288
SYSCTL
Generated by autotune
net.inet.tcp.reass.maxqueuelen
4096
SYSCTL
net.inet.tcp.mssdflt
1448
SYSCTL
net.inet.tcp.delayed_ack
0
SYSCTL
net.add_addr_allfibs
0
SYSCTL