SOLVED Extremely Bad Networking Performance TrueNas 12-Core Freenas Mini

Marcelo Ruiz

Dabbler
Joined
Oct 8, 2015
Messages
20
Hi Community,

I have a really bad networking performance in my iXsystems' Freenas Mini using TrueNas Core 12.0-U6.
To be honest, I don't know when this problem started, but I noticed when I was configuring VLANs and testing the network speed inside jails. I followed Lawrence Hardware tutorial on YouTube and later searched here in the forums for bad network performance in jails to find out that I needed to disable hardware acceleration on the NIC, but that did not help at all. I tried booting to TrueNas 12.0 and the problem persisted, even from the root account without unlocking my storage pool. The performance was so bad, that I thought the NICs were fried. I tried replacing the router, then the lan cables, but the problem persisted.
Later, I decided to boot the Mini with Ubuntu 18.06 and, to my surprise, the Lan speed was fine and the connection rock-solid. After that, I created a booteable usb with TrueNas Core 12.0-U6 (to isolate any possible configuration errors in my main install) and booted from it, and trying iperf3 from the root account to my ubuntu machine the performance was horrible again.
Just to be clear, all the network tests were done via wire connections to a lan desktop in the same network. The numbers that I posted bellow show the difference when using exactly the same hardware with exactly the same connections (even the IPs were the same).
I would like to know if anyone might have a clue about what is going on. The Mini is almost unusable with this performance problem and any suggestion will be really appreciated! Thanks for taking time to looking into this!
Marcelo

FreeNas Mini information:

Code:
FreeNas Mini
UEFI Version: C2750D4I P2.90
BMC Version:  0.35.0
Total Memory: 32768 MB
BIOS Version: 2.16.1243

root@freenas:~ # pciconf -lv
igb0@pci0:7:0:0:    class=0x020000 card=0x15331849 chip=0x15338086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
igb1@pci0:8:0:0:    class=0x020000 card=0x15331849 chip=0x15338086 rev=0x03 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet

root@freenas:~ # ifconfig
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=e523bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether d0:50:99:6f:df:b6
    inet 192.168.10.247 netmask 0xffffff00 broadcast 192.168.10.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=1<PERFORMNUD>
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether d0:50:99:6f:df:b7
    inet 192.168.10.248 netmask 0xffffff00 broadcast 192.168.10.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=1<PERFORMNUD>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:7b:7f:df:6d:00
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 20000
    groups: bridge
    nd6 options=1<PERFORMNUD>


iperf3 test from root@freenas to ubuntu desktop:
Code:
root@freenas:~ # iperf3 -c 192.168.10.114 -p 55505 -t 15 -w 2 -O 5 -Z -i 0 -P 20 - N -R -f m
Connecting to host 192.168.10.114, port 55505
Reverse mode, remote host 192.168.10.114 is sending
[  5] local 192.168.10.248 port 52064 connected to 192.168.10.114 port 55505
[  7] local 192.168.10.248 port 52065 connected to 192.168.10.114 port 55505
[  9] local 192.168.10.248 port 52066 connected to 192.168.10.114 port 55505
[ 11] local 192.168.10.248 port 52067 connected to 192.168.10.114 port 55505
[ 13] local 192.168.10.248 port 52068 connected to 192.168.10.114 port 55505
[ 15] local 192.168.10.248 port 52069 connected to 192.168.10.114 port 55505
[ 17] local 192.168.10.248 port 52070 connected to 192.168.10.114 port 55505
[ 19] local 192.168.10.248 port 52071 connected to 192.168.10.114 port 55505
[ 21] local 192.168.10.248 port 52072 connected to 192.168.10.114 port 55505
[ 23] local 192.168.10.248 port 52073 connected to 192.168.10.114 port 55505
[ 25] local 192.168.10.248 port 52074 connected to 192.168.10.114 port 55505
[ 27] local 192.168.10.248 port 52075 connected to 192.168.10.114 port 55505
[ 29] local 192.168.10.248 port 52076 connected to 192.168.10.114 port 55505
[ 31] local 192.168.10.248 port 52077 connected to 192.168.10.114 port 55505
[ 33] local 192.168.10.248 port 52078 connected to 192.168.10.114 port 55505
[ 35] local 192.168.10.248 port 52079 connected to 192.168.10.114 port 55505
[ 37] local 192.168.10.248 port 52080 connected to 192.168.10.114 port 55505
[ 39] local 192.168.10.248 port 52081 connected to 192.168.10.114 port 55505
[ 41] local 192.168.10.248 port 52082 connected to 192.168.10.114 port 55505
[ 43] local 192.168.10.248 port 52083 connected to 192.168.10.114 port 55505
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[  7]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[  9]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 11]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 13]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 15]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 17]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 19]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 21]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 23]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 25]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 27]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 29]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 31]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 33]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 35]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 37]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 39]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 41]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 43]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[SUM]   0.00-15.00  sec  90.0 KBytes  0.05 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[  5]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[  7]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[  7]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[  9]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[  9]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 11]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 11]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 13]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 13]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 15]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 15]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 17]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 17]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 19]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 19]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 21]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 21]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 23]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 23]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 25]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 25]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 27]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 27]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 29]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 29]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 31]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 31]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 33]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 33]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 35]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 35]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 37]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 37]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 39]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 39]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 41]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 41]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 43]   0.00-15.37  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 43]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[SUM]   0.00-15.37  sec  92.5 KBytes  0.05 Mbits/sec    0             sender
[SUM]   0.00-15.00  sec  90.0 KBytes  0.05 Mbits/sec                  receiver

iperf Done.


Now, same thing with Ubuntu 18.06.
Network details:
Code:
ubuntu@ubuntu:~$ ifconfig
enp7s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.10.247  netmask 255.255.255.0  broadcast 192.168.10.255
        inet6 fe80::269:489e:7284:d6cb  prefixlen 64  scopeid 0x20<link>
        inet6 fda2:ed41:f336:0:f89a:bf46:bda:cd0f  prefixlen 64  scopeid 0x0<global>
        inet6 fda2:ed41:f336::5d3  prefixlen 128  scopeid 0x0<global>
        inet6 fda2:ed41:f336:0:d616:ea47:8a2f:4c8e  prefixlen 64  scopeid 0x0<global>
        ether d0:50:99:6f:df:b6  txqueuelen 1000  (Ethernet)
        RX packets 5818905  bytes 2876862548 (2.8 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5085851  bytes 1194608673 (1.1 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xdf300000-df37ffff 

enp8s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.10.248  netmask 255.255.255.0  broadcast 192.168.10.255
        inet6 fda2:ed41:f336::33f  prefixlen 128  scopeid 0x0<global>
        inet6 fda2:ed41:f336:0:2fc7:996:e6b:a749  prefixlen 64  scopeid 0x0<global>
        inet6 fda2:ed41:f336:0:ad71:101e:ae84:3d9c  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::d220:ba54:a5e6:b7d  prefixlen 64  scopeid 0x20<link>
        ether d0:50:99:6f:df:b7  txqueuelen 1000  (Ethernet)
        RX packets 11590104  bytes 6714689488 (6.7 GB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10065957  bytes 1392856700 (1.3 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device memory 0xdf200000-df27ffff 

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 444  bytes 35635 (35.6 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 444  bytes 35635 (35.6 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


And the iperf3 test:
Code:
ubuntu@ubuntu:~$ iperf3 -c 192.168.10.114 -p 55505 -t 15 -w 2 -O 5 -Z -i 0 -P 20 - N -R -f m
Connecting to host 192.168.10.114, port 55505
Reverse mode, remote host 192.168.10.114 is sending
[  4] local 192.168.10.248 port 43360 connected to 192.168.10.114 port 55505
[  6] local 192.168.10.248 port 43362 connected to 192.168.10.114 port 55505
[  8] local 192.168.10.248 port 43364 connected to 192.168.10.114 port 55505
[ 10] local 192.168.10.248 port 43366 connected to 192.168.10.114 port 55505
[ 12] local 192.168.10.248 port 43368 connected to 192.168.10.114 port 55505
[ 14] local 192.168.10.248 port 43370 connected to 192.168.10.114 port 55505
[ 16] local 192.168.10.248 port 43372 connected to 192.168.10.114 port 55505
[ 18] local 192.168.10.248 port 43374 connected to 192.168.10.114 port 55505
[ 20] local 192.168.10.248 port 43376 connected to 192.168.10.114 port 55505
[ 22] local 192.168.10.248 port 43378 connected to 192.168.10.114 port 55505
[ 24] local 192.168.10.248 port 43380 connected to 192.168.10.114 port 55505
[ 26] local 192.168.10.248 port 43382 connected to 192.168.10.114 port 55505
[ 28] local 192.168.10.248 port 43384 connected to 192.168.10.114 port 55505
[ 30] local 192.168.10.248 port 43386 connected to 192.168.10.114 port 55505
[ 32] local 192.168.10.248 port 43388 connected to 192.168.10.114 port 55505
[ 34] local 192.168.10.248 port 43390 connected to 192.168.10.114 port 55505
[ 36] local 192.168.10.248 port 43392 connected to 192.168.10.114 port 55505
[ 38] local 192.168.10.248 port 43394 connected to 192.168.10.114 port 55505
[ 40] local 192.168.10.248 port 43396 connected to 192.168.10.114 port 55505
[ 42] local 192.168.10.248 port 43398 connected to 192.168.10.114 port 55505
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-15.00  sec  34.4 MBytes  19.2 Mbits/sec                 
[  6]   0.00-15.00  sec  33.5 MBytes  18.7 Mbits/sec                 
[  8]   0.00-15.00  sec  32.7 MBytes  18.3 Mbits/sec                 
[ 10]   0.00-15.00  sec  32.1 MBytes  18.0 Mbits/sec                 
[ 12]   0.00-15.00  sec  31.5 MBytes  17.6 Mbits/sec                 
[ 14]   0.00-15.00  sec  31.2 MBytes  17.5 Mbits/sec                 
[ 16]   0.00-15.00  sec  30.9 MBytes  17.3 Mbits/sec                 
[ 18]   0.00-15.00  sec  30.6 MBytes  17.1 Mbits/sec                 
[ 20]   0.00-15.00  sec  30.2 MBytes  16.9 Mbits/sec                 
[ 22]   0.00-15.00  sec  29.9 MBytes  16.7 Mbits/sec                 
[ 24]   0.00-15.00  sec  29.6 MBytes  16.6 Mbits/sec                 
[ 26]   0.00-15.00  sec  29.4 MBytes  16.4 Mbits/sec                 
[ 28]   0.00-15.00  sec  28.9 MBytes  16.2 Mbits/sec                 
[ 30]   0.00-15.00  sec  28.8 MBytes  16.1 Mbits/sec                 
[ 32]   0.00-15.00  sec  28.4 MBytes  15.9 Mbits/sec                 
[ 34]   0.00-15.00  sec  27.9 MBytes  15.6 Mbits/sec                 
[ 36]   0.00-15.00  sec  27.3 MBytes  15.3 Mbits/sec                 
[ 38]   0.00-15.00  sec  26.8 MBytes  15.0 Mbits/sec                 
[ 40]   0.00-15.00  sec  26.3 MBytes  14.7 Mbits/sec                 
[ 42]   0.00-15.00  sec  25.7 MBytes  14.4 Mbits/sec                 
[SUM]   0.00-15.00  sec   596 MBytes   333 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-15.00  sec  34.4 MBytes  19.2 Mbits/sec    0             sender
[  4]   0.00-15.00  sec  34.4 MBytes  19.2 Mbits/sec                  receiver
[  6]   0.00-15.00  sec  33.5 MBytes  18.7 Mbits/sec    0             sender
[  6]   0.00-15.00  sec  33.5 MBytes  18.7 Mbits/sec                  receiver
[  8]   0.00-15.00  sec  32.7 MBytes  18.3 Mbits/sec    0             sender
[  8]   0.00-15.00  sec  32.7 MBytes  18.3 Mbits/sec                  receiver
[ 10]   0.00-15.00  sec  32.1 MBytes  18.0 Mbits/sec    0             sender
[ 10]   0.00-15.00  sec  32.1 MBytes  18.0 Mbits/sec                  receiver
[ 12]   0.00-15.00  sec  31.5 MBytes  17.6 Mbits/sec    0             sender
[ 12]   0.00-15.00  sec  31.5 MBytes  17.6 Mbits/sec                  receiver
[ 14]   0.00-15.00  sec  31.2 MBytes  17.5 Mbits/sec    0             sender
[ 14]   0.00-15.00  sec  31.2 MBytes  17.5 Mbits/sec                  receiver
[ 16]   0.00-15.00  sec  30.9 MBytes  17.3 Mbits/sec    0             sender
[ 16]   0.00-15.00  sec  30.9 MBytes  17.3 Mbits/sec                  receiver
[ 18]   0.00-15.00  sec  30.6 MBytes  17.1 Mbits/sec    0             sender
[ 18]   0.00-15.00  sec  30.6 MBytes  17.1 Mbits/sec                  receiver
[ 20]   0.00-15.00  sec  30.2 MBytes  16.9 Mbits/sec    0             sender
[ 20]   0.00-15.00  sec  30.2 MBytes  16.9 Mbits/sec                  receiver
[ 22]   0.00-15.00  sec  29.9 MBytes  16.7 Mbits/sec    0             sender
[ 22]   0.00-15.00  sec  29.9 MBytes  16.7 Mbits/sec                  receiver
[ 24]   0.00-15.00  sec  29.6 MBytes  16.6 Mbits/sec    0             sender
[ 24]   0.00-15.00  sec  29.6 MBytes  16.6 Mbits/sec                  receiver
[ 26]   0.00-15.00  sec  29.4 MBytes  16.4 Mbits/sec    0             sender
[ 26]   0.00-15.00  sec  29.4 MBytes  16.4 Mbits/sec                  receiver
[ 28]   0.00-15.00  sec  28.9 MBytes  16.2 Mbits/sec    0             sender
[ 28]   0.00-15.00  sec  28.9 MBytes  16.2 Mbits/sec                  receiver
[ 30]   0.00-15.00  sec  28.8 MBytes  16.1 Mbits/sec    0             sender
[ 30]   0.00-15.00  sec  28.8 MBytes  16.1 Mbits/sec                  receiver
[ 32]   0.00-15.00  sec  28.4 MBytes  15.9 Mbits/sec    0             sender
[ 32]   0.00-15.00  sec  28.4 MBytes  15.9 Mbits/sec                  receiver
[ 34]   0.00-15.00  sec  27.9 MBytes  15.6 Mbits/sec    0             sender
[ 34]   0.00-15.00  sec  27.9 MBytes  15.6 Mbits/sec                  receiver
[ 36]   0.00-15.00  sec  27.3 MBytes  15.3 Mbits/sec    0             sender
[ 36]   0.00-15.00  sec  27.3 MBytes  15.3 Mbits/sec                  receiver
[ 38]   0.00-15.00  sec  26.8 MBytes  15.0 Mbits/sec    0             sender
[ 38]   0.00-15.00  sec  26.8 MBytes  15.0 Mbits/sec                  receiver
[ 40]   0.00-15.00  sec  26.3 MBytes  14.7 Mbits/sec    0             sender
[ 40]   0.00-15.00  sec  26.3 MBytes  14.7 Mbits/sec                  receiver
[ 42]   0.00-15.00  sec  25.7 MBytes  14.4 Mbits/sec    0             sender
[ 42]   0.00-15.00  sec  25.7 MBytes  14.4 Mbits/sec                  receiver
[SUM]   0.00-15.00  sec   596 MBytes   333 Mbits/sec    0             sender
[SUM]   0.00-15.00  sec   596 MBytes   333 Mbits/sec                  receiver

iperf Done.


Now TrueNas booted from the pendrive.
Code:
root@truenas:~ # ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether d0:50:99:6f:df:b6
        inet 192.168.10.247 netmask 0xffffff00 broadcast 192.168.10.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether d0:50:99:6f:df:b7
        inet 192.168.10.248 netmask 0xffffff00 broadcast 192.168.10.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 pre


And lastly, the iperf3 speed from the pendrive root account:

Code:
root@truenas:~ # iperf3 -c 192.168.10.114 -p 55505 -t 15 -w 2 -O 5 -Z -i 0 -P 20 - N -R -f m
Connecting to host 192.168.10.114, port 55505
Reverse mode, remote host 192.168.10.114 is sending
[  5] local 192.168.10.247 port 63048 connected to 192.168.10.114 port 55505
[  7] local 192.168.10.247 port 48638 connected to 192.168.10.114 port 55505
[  9] local 192.168.10.247 port 47109 connected to 192.168.10.114 port 55505
[ 11] local 192.168.10.247 port 13614 connected to 192.168.10.114 port 55505
[ 13] local 192.168.10.247 port 16883 connected to 192.168.10.114 port 55505
[ 15] local 192.168.10.247 port 38387 connected to 192.168.10.114 port 55505
[ 17] local 192.168.10.247 port 61067 connected to 192.168.10.114 port 55505
[ 19] local 192.168.10.247 port 18351 connected to 192.168.10.114 port 55505
[ 21] local 192.168.10.247 port 21696 connected to 192.168.10.114 port 55505
[ 23] local 192.168.10.247 port 16580 connected to 192.168.10.114 port 55505
[ 25] local 192.168.10.247 port 49318 connected to 192.168.10.114 port 55505
[ 27] local 192.168.10.247 port 25154 connected to 192.168.10.114 port 55505
[ 29] local 192.168.10.247 port 19470 connected to 192.168.10.114 port 55505
[ 31] local 192.168.10.247 port 17518 connected to 192.168.10.114 port 55505
[ 33] local 192.168.10.247 port 61725 connected to 192.168.10.114 port 55505
[ 35] local 192.168.10.247 port 50566 connected to 192.168.10.114 port 55505
[ 37] local 192.168.10.247 port 32191 connected to 192.168.10.114 port 55505
[ 39] local 192.168.10.247 port 18225 connected to 192.168.10.114 port 55505
[ 41] local 192.168.10.247 port 17374 connected to 192.168.10.114 port 55505
[ 43] local 192.168.10.247 port 47028 connected to 192.168.10.114 port 55505
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[  7]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[  9]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 11]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 13]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 15]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 17]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 19]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 21]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 23]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 25]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 27]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 29]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 31]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 33]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 35]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 37]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 39]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 41]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[ 43]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                 
[SUM]   0.00-15.00  sec   445 KBytes  0.24 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[  5]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[  7]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[  7]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[  9]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[  9]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 11]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 11]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 13]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 13]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 15]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 15]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 17]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 17]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 19]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 19]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 21]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 21]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 23]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 23]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 25]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 25]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 27]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 27]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 29]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 29]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 31]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 31]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 33]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 33]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 35]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 35]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 37]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 37]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 39]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 39]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 41]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 41]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[ 43]   0.00-15.36  sec  22.8 KBytes  0.01 Mbits/sec    0             sender
[ 43]   0.00-15.00  sec  22.2 KBytes  0.01 Mbits/sec                  receiver
[SUM]   0.00-15.36  sec   455 KBytes  0.24 Mbits/sec    0             sender
[SUM]   0.00-15.00  sec   445 KBytes  0.24 Mbits/sec                  receiver

iperf Done.
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,740
Have you disabled hardware offloading in the physical interface's settings?
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
I think the problem is your setup. You've got igb0 running on 192.168.10.247 and igb1 running on 192.168.10.248. I have the exact same NICs on my server, and I get full rate. What's the performance like with igb1 shutdown? I suspect your setup is resulting in excessive collisions between both NICs. This isn't the proper method for creating a NIC team on TrueNAS.

To have both NICs active in the same subnet, you'd need to create a link aggregation of type either load-balance or round robin, as described in the manual: https://www.truenas.com/docs/core/network/interfaces/laggcreate/. The physical NICs would no longer have an IP, only the lagg0 interface. Any bridge interfaces would also be bound to the lagg interface instead of the physical interfaces.
 
Last edited:

Marcelo Ruiz

Dabbler
Joined
Oct 8, 2015
Messages
20
@Patrick M. Hausen
I did the test disabling hardware offloading when I was working on the jails (following your suggestion in another post) and there was not difference in the test. I did not disable hardware offloading when I booted from the new TrueNas pendrive because I assumed (maybe wrongly?) it was related to jail performance. In both of the tests I posted, all the jails were down (my main pool wasn't unlocked).
I was trying to create a FreeNas 11 booteable pendrive to do the tests with it and see if there were any problems, but I cannot create it: the installer fails with a write error (but the funny thing is that it works for TrueNas 12). I will try to recreate the pendrive for TrueNas 12 and disable hardware offloading and do the test again without starting any jails or anything else.

@Samuel Tai
I have tried with only one interface up (I disconnected a cable) and the results were the same. My router doesn't support link aggregation, so I never used it, but I always had the Mini connected with both Lan cables (since I bought it a couple of years ago) and this was never an issue (an the performance was great).
Another thing that puzzled me when I was trying to troubleshoot the problem using the root console (after connecting a keyboard and a monitor to the Mini) and I deleted the interfaces igb0 and igb1, they were both recreated by the system and both were set to DHCP (which is also the case on a clean install), but this is forbidden when configuring the interfaces via the GUI.
 

Marcelo Ruiz

Dabbler
Joined
Oct 8, 2015
Messages
20
@Patrick M. Hausen
I disabled hardware offloading on both igb0 and igb1, then I rebooted the Mini and performed the iperf3 test with only 1 cable connected (igb0). Here is the result:
Code:
root@freenas:~ # iperf3 -c 192.168.10.114 -p 55505 -t 15 -w 2 -O 5 -Z -i 0 -P 20 - N -R -f m
Connecting to host 192.168.10.114, port 55505
Reverse mode, remote host 192.168.10.114 is sending
[  5] local 192.168.10.247 port 15459 connected to 192.168.10.114 port 55505
[  7] local 192.168.10.247 port 15460 connected to 192.168.10.114 port 55505
[  9] local 192.168.10.247 port 15461 connected to 192.168.10.114 port 55505
[ 11] local 192.168.10.247 port 15462 connected to 192.168.10.114 port 55505
[ 13] local 192.168.10.247 port 15463 connected to 192.168.10.114 port 55505
[ 15] local 192.168.10.247 port 15464 connected to 192.168.10.114 port 55505
[ 17] local 192.168.10.247 port 15465 connected to 192.168.10.114 port 55505
[ 19] local 192.168.10.247 port 15466 connected to 192.168.10.114 port 55505
[ 21] local 192.168.10.247 port 15467 connected to 192.168.10.114 port 55505
[ 23] local 192.168.10.247 port 15468 connected to 192.168.10.114 port 55505
[ 25] local 192.168.10.247 port 15469 connected to 192.168.10.114 port 55505
[ 27] local 192.168.10.247 port 15470 connected to 192.168.10.114 port 55505
[ 29] local 192.168.10.247 port 15471 connected to 192.168.10.114 port 55505
[ 31] local 192.168.10.247 port 15472 connected to 192.168.10.114 port 55505
[ 33] local 192.168.10.247 port 15473 connected to 192.168.10.114 port 55505
[ 35] local 192.168.10.247 port 15474 connected to 192.168.10.114 port 55505
[ 37] local 192.168.10.247 port 15475 connected to 192.168.10.114 port 55505
[ 39] local 192.168.10.247 port 15476 connected to 192.168.10.114 port 55505
[ 41] local 192.168.10.247 port 15477 connected to 192.168.10.114 port 55505
[ 43] local 192.168.10.247 port 15478 connected to 192.168.10.114 port 55505
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[  7]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[  9]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 11]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 13]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 15]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 17]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 19]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 21]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 23]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 25]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 27]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 29]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 31]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 33]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 35]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 37]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 39]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 41]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[ 43]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                 
[SUM]   0.00-15.00  sec  90.0 KBytes  0.05 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[  5]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[  7]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[  7]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[  9]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[  9]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 11]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 11]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 13]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 13]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 15]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 15]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 17]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 17]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 19]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 19]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 21]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 21]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 23]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 23]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 25]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 25]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 27]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 27]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 29]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 29]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 31]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 31]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 33]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 33]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 35]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 35]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 37]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 37]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 39]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 39]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 41]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 41]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[ 43]   0.00-15.38  sec  4.62 KBytes  0.00 Mbits/sec    0             sender
[ 43]   0.00-15.00  sec  4.50 KBytes  0.00 Mbits/sec                  receiver
[SUM]   0.00-15.38  sec  92.5 KBytes  0.05 Mbits/sec    0             sender
[SUM]   0.00-15.00  sec  90.0 KBytes  0.05 Mbits/sec                  receiver

iperf Done.


To add to the confusion, I created the booteable usb for FreeNas 11.3-U5 and the results are (both with hardware ofloading enabled and disabled), only one cable connected

Code:
root@freenas[~]# ping 192.168.10.114                                               
PING 192.168.10.114 (192.168.10.114): 56 data bytes
64 bytes from 192.168.10.114: icmp_seq=0 ttl=64 time=0.193 ms
64 bytes from 192.168.10.114: icmp_seq=1 ttl=64 time=0.232 ms
64 bytes from 192.168.10.114: icmp_seq=2 ttl=64 time=0.300 ms
^C
--- 192.168.10.114 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.193/0.242/0.300/0.044 ms
root@freenas[~]# iperf3 -c 192.168.10.114 -p 55505 -t 15 -w 2 -O 5 -Z -i 0 -P 20 - N -R -f m
Connecting to host 192.168.10.114, port 55505
Reverse mode, remote host 192.168.10.114 is sending
[  5] local 192.168.10.247 port 48306 connected to 192.168.10.114 port 55505
[  7] local 192.168.10.247 port 48307 connected to 192.168.10.114 port 55505
[  9] local 192.168.10.247 port 48308 connected to 192.168.10.114 port 55505
[ 11] local 192.168.10.247 port 48309 connected to 192.168.10.114 port 55505
[ 13] local 192.168.10.247 port 48310 connected to 192.168.10.114 port 55505
[ 15] local 192.168.10.247 port 48311 connected to 192.168.10.114 port 55505
[ 17] local 192.168.10.247 port 48312 connected to 192.168.10.114 port 55505
[ 19] local 192.168.10.247 port 48313 connected to 192.168.10.114 port 55505
[ 21] local 192.168.10.247 port 48314 connected to 192.168.10.114 port 55505
[ 23] local 192.168.10.247 port 48315 connected to 192.168.10.114 port 55505
[ 25] local 192.168.10.247 port 48316 connected to 192.168.10.114 port 55505
[ 27] local 192.168.10.247 port 48317 connected to 192.168.10.114 port 55505
[ 29] local 192.168.10.247 port 48318 connected to 192.168.10.114 port 55505
[ 31] local 192.168.10.247 port 48319 connected to 192.168.10.114 port 55505
[ 33] local 192.168.10.247 port 48320 connected to 192.168.10.114 port 55505
[ 35] local 192.168.10.247 port 48321 connected to 192.168.10.114 port 55505
[ 37] local 192.168.10.247 port 48322 connected to 192.168.10.114 port 55505
[ 39] local 192.168.10.247 port 48323 connected to 192.168.10.114 port 55505
[ 41] local 192.168.10.247 port 48324 connected to 192.168.10.114 port 55505
[ 43] local 192.168.10.247 port 48325 connected to 192.168.10.114 port 55505
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[  7]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[  9]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 11]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 13]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 15]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 17]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 19]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 21]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 23]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 25]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 27]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 29]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 31]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 33]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 35]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 37]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 39]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 41]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[ 43]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
[SUM]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[  5]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[  7]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[  7]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[  9]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[  9]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 11]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 11]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 13]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 13]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 15]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 15]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 17]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 17]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 19]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 19]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 21]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 21]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 23]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 23]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 25]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 25]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 27]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 27]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 29]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 29]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 31]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 31]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 33]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 33]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 35]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 35]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 37]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 37]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 39]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 39]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 41]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 41]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[ 43]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[ 43]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver
[SUM]   0.00-15.40  sec  0.00 Bytes  0.00 Mbits/sec    0             sender
[SUM]   0.00-14.97  sec  0.00 Bytes  0.00 Mbits/sec                  receiver



I don't know if the following information is related in any way, but just in case:
- When the iperf3 test launches, you can tell is going to be slow because the output showing the working processes created connect to the iperf3 server is very slow too. From Ubuntu running on the Mini to the other Ubuntu machine it is lightning fast.
- My install in the Mini was upgraded from FreeNas 11.3-U5 to TrueNas. I didn't mentioned this because I ruled out an upgrade problem when the test from a clean install in a live USB stick I got the same bad results...
 

Marcelo Ruiz

Dabbler
Joined
Oct 8, 2015
Messages
20
Thanks for all the help. I was able to solve this issue. It seems when you edit a property through the GUI in any of the physical interfaces (like setting a static IP address), the GUI deselects the disable hardware offloading, so each time that I changed a setting, I had that problem. The obvious workaround was to set disable hardware offloading in each interface as the last config change, and make them 1 interface at the time (for some reason disabling on both at the same time didn't work for me).
Anyway, thanks a lot for your help!
 
Top