10Gbe woes - and upgrade time!

N

nKk

Dabbler
Joined
Jan 8, 2018
Messages
42
Try to stop all network interfaces that is not used in the moment with:
ifconfig xxx down

This should remove IP addresses that is not necessary.

On Windows side make the same - check for additional static IP addresses on all NIC's and remove them.
If the problems continues please provide output from:

Windows CMD:
ipconfig all
route print

NAS console:
ifconfig
netstat -rna
 
M

Matthias

Explorer
Joined
Mar 7, 2014
Messages
81
Hi guy's! Busy week so far.. so only could get to it now..

Disconnected everything, rebooted, connected the 10Gbe, used 'ifconfig *** down' on all but ix1, 1 ip listed.

so far so good.

(only connect 1 physical lead to your NAS people, even if it is the 'isolated' IPMI port on the server mobo you bought specifically to be able to remote access in the first #*! place..)

(tried setting IPMI in the uefi bios, no success)

Went on testing with iperf:

Code:
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c -p 5201 -bidir
iperf3: error - unable to connect to server:
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 192.168.1.33 -c -p 5201 -bidir
iperf3: error - unable to connect to server:
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -bidir
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.253 port 50995 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   113 MBytes   951 Mbits/sec
[  4]   1.00-2.00   sec   113 MBytes   948 Mbits/sec
[  4]   2.00-3.00   sec   113 MBytes   949 Mbits/sec
[  4]   3.00-4.00   sec   113 MBytes   949 Mbits/sec
[  4]   4.00-5.00   sec   113 MBytes   949 Mbits/sec
[  4]   5.00-6.00   sec   113 MBytes   949 Mbits/sec
[  4]   6.00-7.00   sec   113 MBytes   949 Mbits/sec
[  4]   7.00-8.00   sec   113 MBytes   949 Mbits/sec
[  4]   8.00-9.00   sec   113 MBytes   949 Mbits/sec
[  4]   9.00-10.00  sec   113 MBytes   949 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  1.10 GBytes   949 Mbits/sec                  sender
[  4]   0.00-10.00  sec  1.10 GBytes   949 Mbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -bidir
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.86 port 52020 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   148 MBytes  1.24 Gbits/sec
[  4]   1.00-2.00   sec   146 MBytes  1.22 Gbits/sec
[  4]   2.00-3.00   sec   146 MBytes  1.22 Gbits/sec
[  4]   3.00-4.00   sec   139 MBytes  1.17 Gbits/sec
[  4]   4.00-5.00   sec   141 MBytes  1.18 Gbits/sec
[  4]   5.00-6.00   sec   132 MBytes  1.11 Gbits/sec
[  4]   6.00-7.00   sec   152 MBytes  1.27 Gbits/sec
[  4]   7.00-8.00   sec   137 MBytes  1.15 Gbits/sec
[  4]   8.00-9.00   sec   141 MBytes  1.19 Gbits/sec
[  4]   9.00-10.00  sec   138 MBytes  1.16 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  1.39 GBytes  1.19 Gbits/sec                  sender
[  4]   0.00-10.00  sec  1.39 GBytes  1.19 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -bidir
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.86 port 52027 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   747 MBytes  6.27 Gbits/sec
[  4]   1.00-2.00   sec   750 MBytes  6.29 Gbits/sec
[  4]   2.00-3.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   3.00-4.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   4.00-5.00   sec   724 MBytes  6.07 Gbits/sec
[  4]   5.00-6.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   6.00-7.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   7.00-8.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   8.00-9.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   9.00-10.00  sec   750 MBytes  6.29 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  7.30 GBytes  6.27 Gbits/sec                  sender
[  4]   0.00-10.00  sec  7.30 GBytes  6.27 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64>


So the first measurement is when i still was on my Gbe copper lead on my desktop side. Changed it to 10Gbe fiber after that, and could reach anything, on with the 2-4 iperf test.

Yeah :smile:

tried hooking up the NAS directly to my desktop, no success.

Ifconfig on the NAS side:
Code:
ix0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether 6c:b3:11:0a:24:e0
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4a138b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,NOMAP>
    ether 6c:b3:11:0a:24:e2
    inet 192.168.1.33 netmask 0xffffff00 broadcast 192.168.1.255
    inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
    media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix2: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3b
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3a
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3c
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3d
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 6e:7f:8a:e9:cd:93
    inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 58:9c:fc:10:39:17
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 11 priority 128 path cost 2000
    member: ix1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 2 priority 128 path cost 2000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: PlexJail as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:3d:c3:2d
    hwaddr 02:ad:79:de:36:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.2: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: adguardhomeJail as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:76:4d:54
    hwaddr 02:38:e3:62:b7:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: syncthing as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:5b:07:f8
    hwaddr 02:f3:99:cc:93:0a
    inet 172.16.0.1 netmask 0xfffffffc broadcast 172.16.0.3
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>


ifconfig route print, did not work, but i could see the error 'interface route does not exist' error in the console.

Cutting it to here for today guy's!

To be continued..
 
sretalla

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,700
M

Matthias

Explorer
Joined
Mar 7, 2014
Messages
81
sretalla said:
did you mean netstat -r ?
Click to expand...
No, it went like i stated above.
I didn't do netstat, because of first error.

Then i went on re-connecting everything as it should be, and could do all requested commands.

I tried printing the output, but i didn't succeed..

Goddamn IPMI..

Going to try again later this weekend.
 
Last edited:
N

nKk

Dabbler
Joined
Jan 8, 2018
Messages
42
@sretalla ask because you mismatch the commands:
ifconfig -> list network interfaces on BSD/Linux
netstat -rna -> list routing information on BSD/Linux

ipconfig -> list network interfaces on MS Windows
netstat print -> list routing information on MS Windows

Your command:
ifconfig route print
is wrong and you receive expected error from "ifconfig".

Easiest way to eliminate the switch as bottleneck is to connect NAS and PC directly and you don't need to reconfigure anything. When you plug the fiber optic cable in PC and NAS ports directly you should have the link up and running (check the status from the NAS console) and should have access to the same IP address of the NAS as previously.
 
M

Matthias

Explorer
Joined
Mar 7, 2014
Messages
81
Hi all,

Tried all the above once again to find the culprit..
But however did not succeed in getting the NAS and PC connected directly.

It went as follows:

Checked all commands, when everything's up and running:

NAS 'ifconfig':

Code:
ix0: flags=8c22<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether 6c:b3:11:0a:24:e0
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4a138b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,NOMAP>
    ether 6c:b3:11:0a:24:e2
    inet 192.168.1.33 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix2: flags=8c22<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3b
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix3: flags=8c22<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3a
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3c
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8c22<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3d
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
ue0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 6e:7f:8a:e9:cd:93
    nd6 options=9<PERFORMNUD,IFDISABLED>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 58:9c:fc:10:39:17
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 11 priority 128 path cost 2000
    member: ix1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 2 priority 128 path cost 2000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: PlexJail as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:3d:c3:2d
    hwaddr 02:ad:79:de:36:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.2: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: adguardhomeJail as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:76:4d:54
    hwaddr 02:38:e3:62:b7:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: syncthing as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:5b:07:f8
    hwaddr 02:f3:99:cc:93:0a
    inet 172.16.0.1 netmask 0xfffffffc broadcast 172.16.0.3
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>


NAS: 'netstat' (or 'netstat -rna' not quite sure but it gave an output :smile:):

Code:
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.1        UGS         ix1
127.0.0.1          link#7             UHS         lo0
172.16.0.0/30      link#13            U       vnet0.3
172.16.0.1         link#13            UHS         lo0
192.168.1.0/24     link#2             U           ix1
192.168.1.33       link#2             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#7                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#7                        U           lo0
fe80::1%lo0                       link#7                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0


PC 'ipconfig':
Code:
Windows IP Configuration


Ethernet adapter Ethernet:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Ethernet 5:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Ethernet 4:

   Connection-specific DNS Suffix  . :
   IPv6 Address. . . . . . . . . . . : 2a02:1811:9dba:8400:8ab6:6aa0:8c84:a25c
   Temporary IPv6 Address. . . . . . : 2a02:1811:9dba:8400:b155:d68b:7bcd:12c4
   Link-local IPv6 Address . . . . . : fe80::60e0:623:357e:602a%10
   IPv4 Address. . . . . . . . . . . : 192.168.1.175
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : fe80::6d9:f5ff:fee6:4320%10
                                       192.168.1.1

Ethernet adapter Ethernet 3:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter Wi-Fi:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter LAN-verbinding* 1:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter LAN-verbinding* 2:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter VMware Network Adapter VMnet1:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::12bf:4556:8040:1b7a%14
   IPv4 Address. . . . . . . . . . . : 192.168.241.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter VMware Network Adapter VMnet8:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::c8ee:4faa:51ad:d60f%8
   IPv4 Address. . . . . . . . . . . : 192.168.112.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter Bluetooth-netwerkverbinding:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :


PC 'route print':
Code:
===========================================================================
Interface List
 12...f0 2f 74 b3 9d 89 ......Intel(R) I211 Gigabit Network Connection
 13...6c b3 11 65 78 36 ......Intel(R) Ethernet Server Adapter X520-2 #2
 10...6c b3 11 65 78 34 ......Intel(R) Ethernet Server Adapter X520-2
 20...00 ff c1 c9 24 1b ......TAP-NordVPN Windows Adapter V9
  4...44 af 28 a2 73 80 ......Intel(R) Wi-Fi 6 AX200 160MHz
  9...44 af 28 a2 73 81 ......Microsoft Wi-Fi Direct Virtual Adapter
 21...46 af 28 a2 73 80 ......Microsoft Wi-Fi Direct Virtual Adapter #2
 14...00 50 56 c0 00 01 ......VMware Virtual Ethernet Adapter for VMnet1
  8...00 50 56 c0 00 08 ......VMware Virtual Ethernet Adapter for VMnet8
 25...44 af 28 a2 73 84 ......Bluetooth Device (Personal Area Network)
  1...........................Software Loopback Interface 1
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      192.168.1.1    192.168.1.175     15
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    331
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    331
      192.168.1.0    255.255.255.0         On-link     192.168.1.175    271
    192.168.1.175  255.255.255.255         On-link     192.168.1.175    271
    192.168.1.255  255.255.255.255         On-link     192.168.1.175    271
      192.168.2.0    255.255.255.0     192.168.1.52    192.168.1.175     16
    192.168.112.0    255.255.255.0         On-link     192.168.112.1    291
    192.168.112.1  255.255.255.255         On-link     192.168.112.1    291
  192.168.112.255  255.255.255.255         On-link     192.168.112.1    291
    192.168.241.0    255.255.255.0         On-link     192.168.241.1    291
    192.168.241.1  255.255.255.255         On-link     192.168.241.1    291
  192.168.241.255  255.255.255.255         On-link     192.168.241.1    291
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    331
        224.0.0.0        240.0.0.0         On-link     192.168.241.1    291
        224.0.0.0        240.0.0.0         On-link     192.168.112.1    291
        224.0.0.0        240.0.0.0         On-link     192.168.1.175    271
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    331
  255.255.255.255  255.255.255.255         On-link     192.168.241.1    291
  255.255.255.255  255.255.255.255         On-link     192.168.112.1    291
  255.255.255.255  255.255.255.255         On-link     192.168.1.175    271
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
     192.168.2.20    255.255.255.0     192.168.1.52       1
    192.168.2.252    255.255.255.0     192.168.1.52       1
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
 10    271 ::/0                     fe80::6d9:f5ff:fee6:4320
  1    331 ::1/128                  On-link
 10    271 2a02:1811:9dba:8400::/56 fe80::6d9:f5ff:fee6:4320
 10     31 2a02:1811:9dba:8400::/64 On-link
 10    271 2a02:1811:9dba:8400:8ab6:6aa0:8c84:a25c/128
                                    On-link
 10    271 2a02:1811:9dba:8400:b155:d68b:7bcd:12c4/128
                                    On-link
 14    291 fe80::/64                On-link
  8    291 fe80::/64                On-link
 10    271 fe80::/64                On-link
 14    291 fe80::12bf:4556:8040:1b7a/128
                                    On-link
 10    271 fe80::60e0:623:357e:602a/128
                                    On-link
  8    291 fe80::c8ee:4faa:51ad:d60f/128
                                    On-link
  1    331 ff00::/8                 On-link
 14    291 ff00::/8                 On-link
  8    291 ff00::/8                 On-link
 10    271 ff00::/8                 On-link
===========================================================================
Persistent Routes:
  None


Next up, connected the module from my PC -> SWITCH, directly into the NAS.
Couldn't connect to the NAS IP.
Checked the Intel Proset adapter utility for the uplink itself, it was ok. (green light 10Gbe uplink)
Swapped out the modules again, no luck. Re-plugged the best combo.

re-did all the commands:

NAS ifconfig:
Code:
ix0: flags=8c22<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether 6c:b3:11:0a:24:e0
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4a138b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,NOMAP>
    ether 6c:b3:11:0a:24:e2
    inet 192.168.1.33 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix2: flags=8c22<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3b
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix3: flags=8c22<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3a
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3c
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8c22<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3d
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
ue0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 6e:7f:8a:e9:cd:93
    nd6 options=9<PERFORMNUD,IFDISABLED>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 58:9c:fc:10:39:17
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 11 priority 128 path cost 2000
    member: ix1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 2 priority 128 path cost 2000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: PlexJail as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:3d:c3:2d
    hwaddr 02:ad:79:de:36:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.2: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: adguardhomeJail as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:76:4d:54
    hwaddr 02:38:e3:62:b7:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: syncthing as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:5b:07:f8
    hwaddr 02:f3:99:cc:93:0a
    inet 172.16.0.1 netmask 0xfffffffc broadcast 172.16.0.3
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>


NAS netstat -rna (here i'm pretty sure, 'netstat' only wouldn't give an output):
Code:
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.1        UGS         ix1
127.0.0.1          link#7             UHS         lo0
172.16.0.0/30      link#13            U       vnet0.3
172.16.0.1         link#13            UHS         lo0
192.168.1.0/24     link#2             U           ix1
192.168.1.33       link#2             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#7                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#7                        U           lo0
fe80::1%lo0                       link#7                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0


PC ipconfig:
Code:
Windows IP Configuration


Ethernet adapter Ethernet:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Ethernet 5:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Ethernet 4:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::60e0:623:357e:602a%10
   Autoconfiguration IPv4 Address. . : 169.254.149.66
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . :

Ethernet adapter Ethernet 3:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter Wi-Fi:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter LAN-verbinding* 1:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter LAN-verbinding* 2:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter VMware Network Adapter VMnet1:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::12bf:4556:8040:1b7a%14
   IPv4 Address. . . . . . . . . . . : 192.168.241.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter VMware Network Adapter VMnet8:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::c8ee:4faa:51ad:d60f%8
   IPv4 Address. . . . . . . . . . . : 192.168.112.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter Bluetooth-netwerkverbinding:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :


PC route print:
Code:
===========================================================================
Interface List
 12...f0 2f 74 b3 9d 89 ......Intel(R) I211 Gigabit Network Connection
 13...6c b3 11 65 78 36 ......Intel(R) Ethernet Server Adapter X520-2 #2
 10...6c b3 11 65 78 34 ......Intel(R) Ethernet Server Adapter X520-2
 20...00 ff c1 c9 24 1b ......TAP-NordVPN Windows Adapter V9
  4...44 af 28 a2 73 80 ......Intel(R) Wi-Fi 6 AX200 160MHz
  9...44 af 28 a2 73 81 ......Microsoft Wi-Fi Direct Virtual Adapter
 21...46 af 28 a2 73 80 ......Microsoft Wi-Fi Direct Virtual Adapter #2
 14...00 50 56 c0 00 01 ......VMware Virtual Ethernet Adapter for VMnet1
  8...00 50 56 c0 00 08 ......VMware Virtual Ethernet Adapter for VMnet8
 25...44 af 28 a2 73 84 ......Bluetooth Device (Personal Area Network)
  1...........................Software Loopback Interface 1
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    331
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    331
      169.254.0.0      255.255.0.0         On-link    169.254.149.66    271
   169.254.149.66  255.255.255.255         On-link    169.254.149.66    271
  169.254.255.255  255.255.255.255         On-link    169.254.149.66    271
    192.168.112.0    255.255.255.0         On-link     192.168.112.1    291
    192.168.112.1  255.255.255.255         On-link     192.168.112.1    291
  192.168.112.255  255.255.255.255         On-link     192.168.112.1    291
    192.168.241.0    255.255.255.0         On-link     192.168.241.1    291
    192.168.241.1  255.255.255.255         On-link     192.168.241.1    291
  192.168.241.255  255.255.255.255         On-link     192.168.241.1    291
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    331
        224.0.0.0        240.0.0.0         On-link     192.168.241.1    291
        224.0.0.0        240.0.0.0         On-link     192.168.112.1    291
        224.0.0.0        240.0.0.0         On-link    169.254.149.66    271
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    331
  255.255.255.255  255.255.255.255         On-link     192.168.241.1    291
  255.255.255.255  255.255.255.255         On-link     192.168.112.1    291
  255.255.255.255  255.255.255.255         On-link    169.254.149.66    271
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
     192.168.2.20    255.255.255.0     192.168.1.52       1
    192.168.2.252    255.255.255.0     192.168.1.52       1
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
  1    331 ::1/128                  On-link
 14    291 fe80::/64                On-link
  8    291 fe80::/64                On-link
 10    271 fe80::/64                On-link
 14    291 fe80::12bf:4556:8040:1b7a/128
                                    On-link
 10    271 fe80::60e0:623:357e:602a/128
                                    On-link
  8    291 fe80::c8ee:4faa:51ad:d60f/128
                                    On-link
  1    331 ff00::/8                 On-link
 14    291 ff00::/8                 On-link
  8    291 ff00::/8                 On-link
 10    271 ff00::/8                 On-link
===========================================================================
Persistent Routes:
  None



Glad i picked up a bit of navigation through the cli..

After scrolling through the outputs i noticed the following value's to be strange?
- PC IP (192.168.241.1)
- PC still having a persistent route? And they jump ip range?
Persistent Routes:
Network Address Netmask Gateway Address Metric
192.168.2.20 255.255.255.0 192.168.1.52 1
192.168.2.252 255.255.255.0 192.168.1.52 1

Also checked on the Windows 11 side of the story, booted a fresh copy of Windows 10. (reconnected everything to the SWITCH)
iperf show same results, starting in the low 1.2-3Gbe's to 6.5Gbe max throughput after a couple of runs.

So this should have eliminated every setting i did wrong on the Windows 11 side, but still was not able to connect directly tot the NAS nor achieve 10Gbe-ish.
Swapped the leads, but then the 10Gbe uplink in the Intel Proset Utility went dead.

Triple checked the console on the NAS side if the ip was still 192.168.1.33, it was the entire time.

I suspect my pc's IP range of 2 could be the answer? Than again, using the W10 machine with fresh network settings, that eliminates this possibility right?

Running in circles again..
Maybe your trained eyes spot something.

Thank you all
 
N

nKk

Dabbler
Joined
Jan 8, 2018
Messages
42
I think that the mystery is solved ;)

When you direct connect NAS and PC the NIC's are fine, the link is up and running, on NAS side you have proper network config but on PC side there is a problem.

Because the PC is configured to use DHCP for IP address assignment, there is no DHCP server on direct link and Windows assign IP address from Automatic Private IP Addressing space (169.254.0.0/16)

Code:
Ethernet adapter Ethernet 4:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::60e0:623:357e:602a%10
   Autoconfiguration IPv4 Address. . : 169.254.149.66
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . :


with this IP address you can't access your NAS at 192.168.1.33

You almost find the clue but you looking on wrong network adapter:
Ethernet adapter VMware Network Adapter VMnet1 -> this is virtual adapter used from VMware

Code:
Ethernet adapter VMware Network Adapter VMnet1:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::12bf:4556:8040:1b7a%14
   IPv4 Address. . . . . . . . . . . : 192.168.241.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :


You need to configure "Ethernet 4" with static IP address on Windows for testing direct connection between NAS and PC.
For example:
IP address -> 192.168.1.22
network mask -> 255.255.255.0

After the tests you can switch back to DHCP configuration on PC.
 
M

Matthias

Explorer
Joined
Mar 7, 2014
Messages
81
nKk said:
I think that the mystery is solved ;)

When you direct connect NAS and PC the NIC's are fine, the link is up and running, on NAS side you have proper network config but on PC side there is a problem.

Because the PC is configured to use DHCP for IP address assignment, there is no DHCP server on direct link and Windows assign IP address from Automatic Private IP Addressing space (169.254.0.0/16)

Code:
Ethernet adapter Ethernet 4:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::60e0:623:357e:602a%10
   Autoconfiguration IPv4 Address. . : 169.254.149.66
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . :


with this IP address you can't access your NAS at 192.168.1.33

You almost find the clue but you looking on wrong network adapter:
Ethernet adapter VMware Network Adapter VMnet1 -> this is virtual adapter used from VMware

Code:
Ethernet adapter VMware Network Adapter VMnet1:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::12bf:4556:8040:1b7a%14
   IPv4 Address. . . . . . . . . . . : 192.168.241.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :


You need to configure "Ethernet 4" with static IP address on Windows for testing direct connection between NAS and PC.
For example:
IP address -> 192.168.1.22
network mask -> 255.255.255.0

After the tests you can switch back to DHCP configuration on PC.
Click to expand...
Thank you, and worked!

But same results:

Code:
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.88 port 59885 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   381 MBytes  3.20 Gbits/sec
[  4]   1.00-2.00   sec   378 MBytes  3.17 Gbits/sec
[  4]   2.00-3.00   sec   380 MBytes  3.18 Gbits/sec
[  4]   3.00-4.00   sec   374 MBytes  3.14 Gbits/sec
[  4]   4.00-5.00   sec   376 MBytes  3.16 Gbits/sec
[  4]   5.00-6.00   sec   377 MBytes  3.17 Gbits/sec
[  4]   6.00-7.00   sec   374 MBytes  3.14 Gbits/sec
[  4]   7.00-8.00   sec   378 MBytes  3.17 Gbits/sec
[  4]   8.00-9.00   sec   376 MBytes  3.15 Gbits/sec
[  4]   9.00-10.00  sec   378 MBytes  3.18 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  3.69 GBytes  3.17 Gbits/sec                  sender
[  4]   0.00-10.00  sec  3.69 GBytes  3.17 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64>
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.88 port 58807 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   718 MBytes  6.02 Gbits/sec
[  4]   1.00-2.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   2.00-3.00   sec   744 MBytes  6.24 Gbits/sec
[  4]   3.00-4.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   4.00-5.00   sec   750 MBytes  6.29 Gbits/sec
[  4]   5.00-6.00   sec   745 MBytes  6.25 Gbits/sec
[  4]   6.00-7.00   sec   752 MBytes  6.31 Gbits/sec
[  4]   7.00-8.00   sec   752 MBytes  6.31 Gbits/sec
[  4]   8.00-9.00   sec   744 MBytes  6.25 Gbits/sec
[  4]   9.00-10.00  sec   751 MBytes  6.30 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  7.28 GBytes  6.26 Gbits/sec                  sender
[  4]   0.00-10.00  sec  7.28 GBytes  6.26 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.88 port 58809 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   732 MBytes  6.14 Gbits/sec
[  4]   1.00-2.00   sec   750 MBytes  6.29 Gbits/sec
[  4]   2.00-3.00   sec   753 MBytes  6.32 Gbits/sec
[  4]   3.00-4.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   4.00-5.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   5.00-6.00   sec   746 MBytes  6.26 Gbits/sec
[  4]   6.00-7.00   sec   752 MBytes  6.31 Gbits/sec
[  4]   7.00-8.00   sec   752 MBytes  6.31 Gbits/sec
[  4]   8.00-9.00   sec   751 MBytes  6.30 Gbits/sec
[  4]   9.00-10.00  sec   753 MBytes  6.32 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  7.32 GBytes  6.28 Gbits/sec                  sender
[  4]   0.00-10.00  sec  7.32 GBytes  6.28 Gbits/sec                  receiver

iperf Done.


Then i ran previous commands too, maybe something shows up.

NAS ifconfig:
Code:
ix0: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether 6c:b3:11:0a:24:e0
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4a138b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,NOMAP>
    ether 6c:b3:11:0a:24:e2
    inet 192.168.1.33 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect (10Gbase-SR <full-duplex,rxpause,txpause>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix2: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3b
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
ix3: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3a
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3c
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8822<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
    ether d0:50:99:fe:5b:3d
    media: Ethernet autoselect
    status: no carrier
    nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
    groups: pflog
ue0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 6e:7f:8a:e9:cd:93
    nd6 options=9<PERFORMNUD,IFDISABLED>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 58:9c:fc:10:39:17
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 11 priority 128 path cost 2000
    member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: ix1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 2 priority 128 path cost 2000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.2: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: adguardhomeJail as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:76:4d:54
    hwaddr 02:38:e3:62:b7:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: syncthing as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:5b:07:f8
    hwaddr 02:f3:99:cc:93:0a
    inet 172.16.0.1 netmask 0xfffffffc broadcast 172.16.0.3
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.4: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: PlexJail as nic: epair0b
    options=8<VLAN_MTU>
    ether d2:50:99:3d:c3:2d
    hwaddr 02:51:4a:76:fe:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=9<PERFORMNUD,IFDISABLED>


NAS netstat -rna:
Code:
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.1.1        UGS         ix1
127.0.0.1          link#7             UH          lo0
172.16.0.0/30      link#13            U       vnet0.3
172.16.0.1         link#13            UHS         lo0
192.168.1.0/24     link#2             U           ix1
192.168.1.33       link#2             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#7                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#7                        U           lo0
fe80::1%lo0                       link#7                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0


PC ipconfig:
Code:
Windows IP Configuration


Ethernet adapter Ethernet:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Ethernet 5:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter Ethernet 4:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::60e0:623:357e:602a%10
   IPv4 Address. . . . . . . . . . . : 192.168.1.88
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter Ethernet 3:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter Wi-Fi:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter LAN-verbinding* 1:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Wireless LAN adapter LAN-verbinding* 2:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :

Ethernet adapter VMware Network Adapter VMnet1:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::12bf:4556:8040:1b7a%14
   IPv4 Address. . . . . . . . . . . : 192.168.241.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter VMware Network Adapter VMnet8:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::c8ee:4faa:51ad:d60f%8
   IPv4 Address. . . . . . . . . . . : 192.168.112.1
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . :

Ethernet adapter Bluetooth-netwerkverbinding:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :


PC route print:
Code:
===========================================================================
Interface List
 12...f0 2f 74 b3 9d 89 ......Intel(R) I211 Gigabit Network Connection
 13...6c b3 11 65 78 36 ......Intel(R) Ethernet Server Adapter X520-2 #2
 10...6c b3 11 65 78 34 ......Intel(R) Ethernet Server Adapter X520-2
 20...00 ff c1 c9 24 1b ......TAP-NordVPN Windows Adapter V9
  4...44 af 28 a2 73 80 ......Intel(R) Wi-Fi 6 AX200 160MHz
  9...44 af 28 a2 73 81 ......Microsoft Wi-Fi Direct Virtual Adapter
 21...46 af 28 a2 73 80 ......Microsoft Wi-Fi Direct Virtual Adapter #2
 14...00 50 56 c0 00 01 ......VMware Virtual Ethernet Adapter for VMnet1
  8...00 50 56 c0 00 08 ......VMware Virtual Ethernet Adapter for VMnet8
 25...44 af 28 a2 73 84 ......Bluetooth Device (Personal Area Network)
  1...........................Software Loopback Interface 1
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    331
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    331
      192.168.1.0    255.255.255.0         On-link      192.168.1.88    271
     192.168.1.88  255.255.255.255         On-link      192.168.1.88    271
    192.168.1.255  255.255.255.255         On-link      192.168.1.88    271
      192.168.2.0    255.255.255.0     192.168.1.52     192.168.1.88     16
    192.168.112.0    255.255.255.0         On-link     192.168.112.1    291
    192.168.112.1  255.255.255.255         On-link     192.168.112.1    291
  192.168.112.255  255.255.255.255         On-link     192.168.112.1    291
    192.168.241.0    255.255.255.0         On-link     192.168.241.1    291
    192.168.241.1  255.255.255.255         On-link     192.168.241.1    291
  192.168.241.255  255.255.255.255         On-link     192.168.241.1    291
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    331
        224.0.0.0        240.0.0.0         On-link     192.168.241.1    291
        224.0.0.0        240.0.0.0         On-link     192.168.112.1    291
        224.0.0.0        240.0.0.0         On-link      192.168.1.88    271
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    331
  255.255.255.255  255.255.255.255         On-link     192.168.241.1    291
  255.255.255.255  255.255.255.255         On-link     192.168.112.1    291
  255.255.255.255  255.255.255.255         On-link      192.168.1.88    271
===========================================================================
Persistent Routes:
  Network Address          Netmask  Gateway Address  Metric
     192.168.2.20    255.255.255.0     192.168.1.52       1
    192.168.2.252    255.255.255.0     192.168.1.52       1
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
 If Metric Network Destination      Gateway
  1    331 ::1/128                  On-link
 14    291 fe80::/64                On-link
  8    291 fe80::/64                On-link
 10    271 fe80::/64                On-link
 14    291 fe80::12bf:4556:8040:1b7a/128
                                    On-link
 10    271 fe80::60e0:623:357e:602a/128
                                    On-link
  8    291 fe80::c8ee:4faa:51ad:d60f/128
                                    On-link
  1    331 ff00::/8                 On-link
 14    291 ff00::/8                 On-link
  8    291 ff00::/8                 On-link
 10    271 ff00::/8                 On-link
===========================================================================
Persistent Routes:
  None


Soo... i was kind of hoping that the switch would be it, but than again not so much XD..

So next investigated with different combo's of SFP modules that work, same ballpark figures again.

Next on the checklist, i'd presume checking the firmware of both cards?
Or drivers?

Thanks again for the tips!
 
M

Matthias

Explorer
Joined
Mar 7, 2014
Messages
81
Or is my PCIe 2.0 x16 card not fully utilising the PCIe 3.0 x4 slot?

Checked the 2.0 specs, it runs at 4Gb/s per lane so x4 should be ok i think unless the change in PCIe spec creates overhead?
 
N

nKk

Dabbler
Joined
Jan 8, 2018
Messages
42
I think the PCIe version is not the problem.
I make some tests here on virtual Windows 10 and the strange thing is that I receive almost the same 6Gbps when I execute:

Code:
iperf3.exe -c 10.10.10.10

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   760 MBytes  6.37 Gbits/sec
[  4]   1.00-2.00   sec   760 MBytes  6.38 Gbits/sec
[  4]   2.00-3.00   sec   774 MBytes  6.49 Gbits/sec
[  4]   3.00-4.00   sec   683 MBytes  5.73 Gbits/sec
[  4]   4.00-5.00   sec   772 MBytes  6.48 Gbits/sec
[  4]   5.00-6.00   sec   746 MBytes  6.25 Gbits/sec
[  4]   6.00-7.00   sec   733 MBytes  6.15 Gbits/sec
[  4]   7.00-8.00   sec   817 MBytes  6.85 Gbits/sec
[  4]   8.00-9.00   sec   735 MBytes  6.17 Gbits/sec
[  4]   9.00-10.00  sec   746 MBytes  6.26 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  7.35 GBytes  6.31 Gbits/sec                  sender
[  4]   0.00-10.00  sec  7.35 GBytes  6.31 Gbits/sec                  receiver


After this I try with iperf3 "-P" options for multiple streams:
Code:
iperf3.exe -c 10.10.10.10 -P 4
...
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  4.28 GBytes  3.67 Gbits/sec                  sender
[  4]   0.00-10.00  sec  4.28 GBytes  3.67 Gbits/sec                  receiver
[  6]   0.00-10.00  sec  3.81 GBytes  3.27 Gbits/sec                  sender
[  6]   0.00-10.00  sec  3.81 GBytes  3.27 Gbits/sec                  receiver
[  8]   0.00-10.00  sec  4.09 GBytes  3.51 Gbits/sec                  sender
[  8]   0.00-10.00  sec  4.09 GBytes  3.51 Gbits/sec                  receiver
[ 10]   0.00-10.00  sec  3.00 GBytes  2.58 Gbits/sec                  sender
[ 10]   0.00-10.00  sec  3.00 GBytes  2.58 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  15.2 GBytes  13.0 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  15.2 GBytes  13.0 Gbits/sec                  receiver


The SUM of the streams in my case are 15Gbps and on Windows Task Manager/Performance/Ethernet I see the same 15Gbps.

I'm not sure if iperf3 for Windows have limitations and can't reach max speed with one stream (execution without -P) but you can try to use "-P 4"
 
M

Matthias

Explorer
Joined
Mar 7, 2014
Messages
81
nKk said:
I think the PCIe version is not the problem.
I make some tests here on virtual Windows 10 and the strange thing is that I receive almost the same 6Gbps when I execute:

Code:
iperf3.exe -c 10.10.10.10

[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   760 MBytes  6.37 Gbits/sec
[  4]   1.00-2.00   sec   760 MBytes  6.38 Gbits/sec
[  4]   2.00-3.00   sec   774 MBytes  6.49 Gbits/sec
[  4]   3.00-4.00   sec   683 MBytes  5.73 Gbits/sec
[  4]   4.00-5.00   sec   772 MBytes  6.48 Gbits/sec
[  4]   5.00-6.00   sec   746 MBytes  6.25 Gbits/sec
[  4]   6.00-7.00   sec   733 MBytes  6.15 Gbits/sec
[  4]   7.00-8.00   sec   817 MBytes  6.85 Gbits/sec
[  4]   8.00-9.00   sec   735 MBytes  6.17 Gbits/sec
[  4]   9.00-10.00  sec   746 MBytes  6.26 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  7.35 GBytes  6.31 Gbits/sec                  sender
[  4]   0.00-10.00  sec  7.35 GBytes  6.31 Gbits/sec                  receiver


After this I try with iperf3 "-P" options for multiple streams:
Code:
iperf3.exe -c 10.10.10.10 -P 4
...
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  4.28 GBytes  3.67 Gbits/sec                  sender
[  4]   0.00-10.00  sec  4.28 GBytes  3.67 Gbits/sec                  receiver
[  6]   0.00-10.00  sec  3.81 GBytes  3.27 Gbits/sec                  sender
[  6]   0.00-10.00  sec  3.81 GBytes  3.27 Gbits/sec                  receiver
[  8]   0.00-10.00  sec  4.09 GBytes  3.51 Gbits/sec                  sender
[  8]   0.00-10.00  sec  4.09 GBytes  3.51 Gbits/sec                  receiver
[ 10]   0.00-10.00  sec  3.00 GBytes  2.58 Gbits/sec                  sender
[ 10]   0.00-10.00  sec  3.00 GBytes  2.58 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  15.2 GBytes  13.0 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  15.2 GBytes  13.0 Gbits/sec                  receiver


The SUM of the streams in my case are 15Gbps and on Windows Task Manager/Performance/Ethernet I see the same 15Gbps.

I'm not sure if iperf3 for Windows have limitations and can't reach max speed with one stream (execution without -P) but you can try to use "-P 4"
Click to expand...
Yup, that's it!

Following results, through the switch:

Code:
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -P 4
iperf3: error - unable to connect to server: Connection refused
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -P 4
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.175 port 50438 connected to 192.168.1.33 port 5201
[  6] local 192.168.1.175 port 50439 connected to 192.168.1.33 port 5201
[  8] local 192.168.1.175 port 50440 connected to 192.168.1.33 port 5201
[ 10] local 192.168.1.175 port 50441 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   155 MBytes  1.30 Gbits/sec
[  6]   0.00-1.00   sec   140 MBytes  1.18 Gbits/sec
[  8]   0.00-1.00   sec   152 MBytes  1.27 Gbits/sec
[ 10]   0.00-1.00   sec   166 MBytes  1.39 Gbits/sec
[SUM]   0.00-1.00   sec   613 MBytes  5.14 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.00-2.00   sec   161 MBytes  1.35 Gbits/sec
[  6]   1.00-2.00   sec   154 MBytes  1.30 Gbits/sec
[  8]   1.00-2.00   sec   157 MBytes  1.32 Gbits/sec
[ 10]   1.00-2.00   sec   168 MBytes  1.41 Gbits/sec
[SUM]   1.00-2.00   sec   640 MBytes  5.37 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec   161 MBytes  1.35 Gbits/sec
[  6]   2.00-3.00   sec   149 MBytes  1.25 Gbits/sec
[  8]   2.00-3.00   sec   162 MBytes  1.36 Gbits/sec
[ 10]   2.00-3.00   sec   161 MBytes  1.35 Gbits/sec
[SUM]   2.00-3.00   sec   633 MBytes  5.31 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec   142 MBytes  1.19 Gbits/sec
[  6]   3.00-4.00   sec   148 MBytes  1.25 Gbits/sec
[  8]   3.00-4.00   sec   149 MBytes  1.25 Gbits/sec
[ 10]   3.00-4.00   sec   160 MBytes  1.34 Gbits/sec
[SUM]   3.00-4.00   sec   600 MBytes  5.03 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec   148 MBytes  1.24 Gbits/sec
[  6]   4.00-5.00   sec   167 MBytes  1.40 Gbits/sec
[  8]   4.00-5.00   sec   156 MBytes  1.31 Gbits/sec
[ 10]   4.00-5.00   sec   168 MBytes  1.41 Gbits/sec
[SUM]   4.00-5.00   sec   639 MBytes  5.36 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec   165 MBytes  1.38 Gbits/sec
[  6]   5.00-6.00   sec   173 MBytes  1.45 Gbits/sec
[  8]   5.00-6.00   sec   152 MBytes  1.28 Gbits/sec
[ 10]   5.00-6.00   sec   172 MBytes  1.44 Gbits/sec
[SUM]   5.00-6.00   sec   662 MBytes  5.55 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec   162 MBytes  1.36 Gbits/sec
[  6]   6.00-7.00   sec   161 MBytes  1.35 Gbits/sec
[  8]   6.00-7.00   sec   150 MBytes  1.26 Gbits/sec
[ 10]   6.00-7.00   sec   151 MBytes  1.27 Gbits/sec
[SUM]   6.00-7.00   sec   625 MBytes  5.24 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec   154 MBytes  1.29 Gbits/sec
[  6]   7.00-8.00   sec   157 MBytes  1.32 Gbits/sec
[  8]   7.00-8.00   sec   162 MBytes  1.36 Gbits/sec
[ 10]   7.00-8.00   sec   165 MBytes  1.39 Gbits/sec
[SUM]   7.00-8.00   sec   638 MBytes  5.35 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec   170 MBytes  1.43 Gbits/sec
[  6]   8.00-9.00   sec   156 MBytes  1.31 Gbits/sec
[  8]   8.00-9.00   sec   161 MBytes  1.35 Gbits/sec
[ 10]   8.00-9.00   sec   162 MBytes  1.36 Gbits/sec
[SUM]   8.00-9.00   sec   649 MBytes  5.44 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec   152 MBytes  1.28 Gbits/sec
[  6]   9.00-10.00  sec   150 MBytes  1.26 Gbits/sec
[  8]   9.00-10.00  sec   178 MBytes  1.50 Gbits/sec
[ 10]   9.00-10.00  sec   168 MBytes  1.41 Gbits/sec
[SUM]   9.00-10.00  sec   649 MBytes  5.44 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  1.53 GBytes  1.32 Gbits/sec                  sender
[  4]   0.00-10.00  sec  1.53 GBytes  1.32 Gbits/sec                  receiver
[  6]   0.00-10.00  sec  1.52 GBytes  1.31 Gbits/sec                  sender
[  6]   0.00-10.00  sec  1.52 GBytes  1.31 Gbits/sec                  receiver
[  8]   0.00-10.00  sec  1.54 GBytes  1.32 Gbits/sec                  sender
[  8]   0.00-10.00  sec  1.54 GBytes  1.32 Gbits/sec                  receiver
[ 10]   0.00-10.00  sec  1.60 GBytes  1.38 Gbits/sec                  sender
[ 10]   0.00-10.00  sec  1.60 GBytes  1.38 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  6.20 GBytes  5.32 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  6.20 GBytes  5.32 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -P 4
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.175 port 50443 connected to 192.168.1.33 port 5201
[  6] local 192.168.1.175 port 50444 connected to 192.168.1.33 port 5201
[  8] local 192.168.1.175 port 50445 connected to 192.168.1.33 port 5201
[ 10] local 192.168.1.175 port 50446 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   277 MBytes  2.32 Gbits/sec
[  6]   0.00-1.00   sec   285 MBytes  2.39 Gbits/sec
[  8]   0.00-1.00   sec   278 MBytes  2.33 Gbits/sec
[ 10]   0.00-1.00   sec   286 MBytes  2.40 Gbits/sec
[SUM]   0.00-1.00   sec  1.10 GBytes  9.45 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.00-2.00   sec   284 MBytes  2.38 Gbits/sec
[  6]   1.00-2.00   sec   284 MBytes  2.38 Gbits/sec
[  8]   1.00-2.00   sec   279 MBytes  2.34 Gbits/sec
[ 10]   1.00-2.00   sec   284 MBytes  2.39 Gbits/sec
[SUM]   1.00-2.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec   284 MBytes  2.38 Gbits/sec
[  6]   2.00-3.00   sec   283 MBytes  2.37 Gbits/sec
[  8]   2.00-3.00   sec   281 MBytes  2.36 Gbits/sec
[ 10]   2.00-3.00   sec   283 MBytes  2.38 Gbits/sec
[SUM]   2.00-3.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec   284 MBytes  2.39 Gbits/sec
[  6]   3.00-4.00   sec   282 MBytes  2.37 Gbits/sec
[  8]   3.00-4.00   sec   280 MBytes  2.35 Gbits/sec
[ 10]   3.00-4.00   sec   284 MBytes  2.38 Gbits/sec
[SUM]   3.00-4.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec   286 MBytes  2.40 Gbits/sec
[  6]   4.00-5.00   sec   291 MBytes  2.44 Gbits/sec
[  8]   4.00-5.00   sec   261 MBytes  2.19 Gbits/sec
[ 10]   4.00-5.00   sec   293 MBytes  2.46 Gbits/sec
[SUM]   4.00-5.00   sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec   285 MBytes  2.39 Gbits/sec
[  6]   5.00-6.00   sec   284 MBytes  2.39 Gbits/sec
[  8]   5.00-6.00   sec   277 MBytes  2.33 Gbits/sec
[ 10]   5.00-6.00   sec   284 MBytes  2.38 Gbits/sec
[SUM]   5.00-6.00   sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec   285 MBytes  2.39 Gbits/sec
[  6]   6.00-7.00   sec   282 MBytes  2.37 Gbits/sec
[  8]   6.00-7.00   sec   279 MBytes  2.34 Gbits/sec
[ 10]   6.00-7.00   sec   285 MBytes  2.39 Gbits/sec
[SUM]   6.00-7.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec   283 MBytes  2.37 Gbits/sec
[  6]   7.00-8.00   sec   282 MBytes  2.37 Gbits/sec
[  8]   7.00-8.00   sec   281 MBytes  2.36 Gbits/sec
[ 10]   7.00-8.00   sec   284 MBytes  2.39 Gbits/sec
[SUM]   7.00-8.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec   284 MBytes  2.39 Gbits/sec
[  6]   8.00-9.00   sec   284 MBytes  2.38 Gbits/sec
[  8]   8.00-9.00   sec   279 MBytes  2.34 Gbits/sec
[ 10]   8.00-9.00   sec   284 MBytes  2.38 Gbits/sec
[SUM]   8.00-9.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec   284 MBytes  2.38 Gbits/sec
[  6]   9.00-10.00  sec   283 MBytes  2.38 Gbits/sec
[  8]   9.00-10.00  sec   279 MBytes  2.34 Gbits/sec
[ 10]   9.00-10.00  sec   285 MBytes  2.39 Gbits/sec
[SUM]   9.00-10.00  sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  2.77 GBytes  2.38 Gbits/sec                  sender
[  4]   0.00-10.00  sec  2.77 GBytes  2.38 Gbits/sec                  receiver
[  6]   0.00-10.00  sec  2.77 GBytes  2.38 Gbits/sec                  sender
[  6]   0.00-10.00  sec  2.77 GBytes  2.38 Gbits/sec                  receiver
[  8]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec                  sender
[  8]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec                  receiver
[ 10]   0.00-10.00  sec  2.79 GBytes  2.39 Gbits/sec                  sender
[ 10]   0.00-10.00  sec  2.79 GBytes  2.39 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  11.0 GBytes  9.48 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  11.0 GBytes  9.48 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -P 4
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.175 port 50449 connected to 192.168.1.33 port 5201
[  6] local 192.168.1.175 port 50450 connected to 192.168.1.33 port 5201
[  8] local 192.168.1.175 port 50451 connected to 192.168.1.33 port 5201
[ 10] local 192.168.1.175 port 50452 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   282 MBytes  2.36 Gbits/sec
[  6]   0.00-1.00   sec   283 MBytes  2.38 Gbits/sec
[  8]   0.00-1.00   sec   283 MBytes  2.37 Gbits/sec
[ 10]   0.00-1.00   sec   281 MBytes  2.36 Gbits/sec
[SUM]   0.00-1.00   sec  1.10 GBytes  9.47 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.00-2.00   sec   278 MBytes  2.33 Gbits/sec
[  6]   1.00-2.00   sec   286 MBytes  2.40 Gbits/sec
[  8]   1.00-2.00   sec   284 MBytes  2.39 Gbits/sec
[ 10]   1.00-2.00   sec   282 MBytes  2.37 Gbits/sec
[SUM]   1.00-2.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec   282 MBytes  2.36 Gbits/sec
[  6]   2.00-3.00   sec   283 MBytes  2.38 Gbits/sec
[  8]   2.00-3.00   sec   284 MBytes  2.38 Gbits/sec
[ 10]   2.00-3.00   sec   283 MBytes  2.37 Gbits/sec
[SUM]   2.00-3.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec   281 MBytes  2.36 Gbits/sec
[  6]   3.00-4.00   sec   282 MBytes  2.37 Gbits/sec
[  8]   3.00-4.00   sec   284 MBytes  2.38 Gbits/sec
[ 10]   3.00-4.00   sec   284 MBytes  2.38 Gbits/sec
[SUM]   3.00-4.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec   255 MBytes  2.14 Gbits/sec
[  6]   4.00-5.00   sec   294 MBytes  2.47 Gbits/sec
[  8]   4.00-5.00   sec   294 MBytes  2.47 Gbits/sec
[ 10]   4.00-5.00   sec   287 MBytes  2.41 Gbits/sec
[SUM]   4.00-5.00   sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec   283 MBytes  2.37 Gbits/sec
[  6]   5.00-6.00   sec   283 MBytes  2.38 Gbits/sec
[  8]   5.00-6.00   sec   283 MBytes  2.37 Gbits/sec
[ 10]   5.00-6.00   sec   282 MBytes  2.36 Gbits/sec
[SUM]   5.00-6.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec   282 MBytes  2.37 Gbits/sec
[  6]   6.00-7.00   sec   283 MBytes  2.38 Gbits/sec
[  8]   6.00-7.00   sec   282 MBytes  2.37 Gbits/sec
[ 10]   6.00-7.00   sec   283 MBytes  2.38 Gbits/sec
[SUM]   6.00-7.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec   281 MBytes  2.36 Gbits/sec
[  6]   7.00-8.00   sec   283 MBytes  2.37 Gbits/sec
[  8]   7.00-8.00   sec   283 MBytes  2.38 Gbits/sec
[ 10]   7.00-8.00   sec   283 MBytes  2.38 Gbits/sec
[SUM]   7.00-8.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec   280 MBytes  2.35 Gbits/sec
[  6]   8.00-9.00   sec   284 MBytes  2.38 Gbits/sec
[  8]   8.00-9.00   sec   284 MBytes  2.38 Gbits/sec
[ 10]   8.00-9.00   sec   283 MBytes  2.38 Gbits/sec
[SUM]   8.00-9.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec   280 MBytes  2.35 Gbits/sec
[  6]   9.00-10.00  sec   283 MBytes  2.38 Gbits/sec
[  8]   9.00-10.00  sec   284 MBytes  2.38 Gbits/sec
[ 10]   9.00-10.00  sec   284 MBytes  2.38 Gbits/sec
[SUM]   9.00-10.00  sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  2.72 GBytes  2.33 Gbits/sec                  sender
[  4]   0.00-10.00  sec  2.72 GBytes  2.33 Gbits/sec                  receiver
[  6]   0.00-10.00  sec  2.78 GBytes  2.39 Gbits/sec                  sender
[  6]   0.00-10.00  sec  2.78 GBytes  2.39 Gbits/sec                  receiver
[  8]   0.00-10.00  sec  2.78 GBytes  2.39 Gbits/sec                  sender
[  8]   0.00-10.00  sec  2.78 GBytes  2.39 Gbits/sec                  receiver
[ 10]   0.00-10.00  sec  2.77 GBytes  2.38 Gbits/sec                  sender
[ 10]   0.00-10.00  sec  2.77 GBytes  2.38 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  11.0 GBytes  9.49 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  11.0 GBytes  9.48 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -P 10
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.175 port 50456 connected to 192.168.1.33 port 5201
[  6] local 192.168.1.175 port 50457 connected to 192.168.1.33 port 5201
[  8] local 192.168.1.175 port 50458 connected to 192.168.1.33 port 5201
[ 10] local 192.168.1.175 port 50459 connected to 192.168.1.33 port 5201
[ 12] local 192.168.1.175 port 50460 connected to 192.168.1.33 port 5201
[ 14] local 192.168.1.175 port 50461 connected to 192.168.1.33 port 5201
[ 16] local 192.168.1.175 port 50462 connected to 192.168.1.33 port 5201
[ 18] local 192.168.1.175 port 50463 connected to 192.168.1.33 port 5201
[ 20] local 192.168.1.175 port 50464 connected to 192.168.1.33 port 5201
[ 22] local 192.168.1.175 port 50465 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   140 MBytes  1.18 Gbits/sec
[  6]   0.00-1.00   sec   140 MBytes  1.18 Gbits/sec
[  8]   0.00-1.00   sec   143 MBytes  1.20 Gbits/sec
[ 10]   0.00-1.00   sec   141 MBytes  1.18 Gbits/sec
[ 12]   0.00-1.00   sec   140 MBytes  1.17 Gbits/sec
[ 14]   0.00-1.00   sec   142 MBytes  1.19 Gbits/sec
[ 16]   0.00-1.00   sec  94.0 MBytes   788 Mbits/sec
[ 18]   0.00-1.00   sec  71.6 MBytes   601 Mbits/sec
[ 20]   0.00-1.00   sec  49.0 MBytes   411 Mbits/sec
[ 22]   0.00-1.00   sec  71.5 MBytes   600 Mbits/sec
[SUM]   0.00-1.00   sec  1.11 GBytes  9.50 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.00-2.00   sec   141 MBytes  1.19 Gbits/sec
[  6]   1.00-2.00   sec   141 MBytes  1.19 Gbits/sec
[  8]   1.00-2.00   sec   144 MBytes  1.20 Gbits/sec
[ 10]   1.00-2.00   sec   136 MBytes  1.14 Gbits/sec
[ 12]   1.00-2.00   sec   138 MBytes  1.16 Gbits/sec
[ 14]   1.00-2.00   sec   138 MBytes  1.15 Gbits/sec
[ 16]   1.00-2.00   sec  73.5 MBytes   617 Mbits/sec
[ 18]   1.00-2.00   sec  63.0 MBytes   529 Mbits/sec
[ 20]   1.00-2.00   sec  73.4 MBytes   616 Mbits/sec
[ 22]   1.00-2.00   sec  83.8 MBytes   703 Mbits/sec
[SUM]   1.00-2.00   sec  1.11 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec   141 MBytes  1.19 Gbits/sec
[  6]   2.00-3.00   sec   142 MBytes  1.19 Gbits/sec
[  8]   2.00-3.00   sec   142 MBytes  1.19 Gbits/sec
[ 10]   2.00-3.00   sec   141 MBytes  1.18 Gbits/sec
[ 12]   2.00-3.00   sec   141 MBytes  1.19 Gbits/sec
[ 14]   2.00-3.00   sec   141 MBytes  1.18 Gbits/sec
[ 16]   2.00-3.00   sec  70.9 MBytes   595 Mbits/sec
[ 18]   2.00-3.00   sec  54.4 MBytes   456 Mbits/sec
[ 20]   2.00-3.00   sec  71.0 MBytes   596 Mbits/sec
[ 22]   2.00-3.00   sec  87.4 MBytes   733 Mbits/sec
[SUM]   2.00-3.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec   142 MBytes  1.19 Gbits/sec
[  6]   3.00-4.00   sec   142 MBytes  1.19 Gbits/sec
[  8]   3.00-4.00   sec   142 MBytes  1.20 Gbits/sec
[ 10]   3.00-4.00   sec   140 MBytes  1.17 Gbits/sec
[ 12]   3.00-4.00   sec   138 MBytes  1.16 Gbits/sec
[ 14]   3.00-4.00   sec   142 MBytes  1.19 Gbits/sec
[ 16]   3.00-4.00   sec  71.4 MBytes   599 Mbits/sec
[ 18]   3.00-4.00   sec  59.5 MBytes   499 Mbits/sec
[ 20]   3.00-4.00   sec  71.2 MBytes   598 Mbits/sec
[ 22]   3.00-4.00   sec  83.2 MBytes   698 Mbits/sec
[SUM]   3.00-4.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec   139 MBytes  1.16 Gbits/sec
[  6]   4.00-5.00   sec   136 MBytes  1.14 Gbits/sec
[  8]   4.00-5.00   sec   151 MBytes  1.27 Gbits/sec
[ 10]   4.00-5.00   sec   124 MBytes  1.04 Gbits/sec
[ 12]   4.00-5.00   sec   124 MBytes  1.04 Gbits/sec
[ 14]   4.00-5.00   sec   149 MBytes  1.25 Gbits/sec
[ 16]   4.00-5.00   sec  78.5 MBytes   659 Mbits/sec
[ 18]   4.00-5.00   sec  69.1 MBytes   580 Mbits/sec
[ 20]   4.00-5.00   sec  78.2 MBytes   656 Mbits/sec
[ 22]   4.00-5.00   sec  82.4 MBytes   691 Mbits/sec
[SUM]   4.00-5.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec   141 MBytes  1.18 Gbits/sec
[  6]   5.00-6.00   sec   140 MBytes  1.18 Gbits/sec
[  8]   5.00-6.00   sec   143 MBytes  1.20 Gbits/sec
[ 10]   5.00-6.00   sec   141 MBytes  1.18 Gbits/sec
[ 12]   5.00-6.00   sec   139 MBytes  1.17 Gbits/sec
[ 14]   5.00-6.00   sec   140 MBytes  1.17 Gbits/sec
[ 16]   5.00-6.00   sec  72.1 MBytes   605 Mbits/sec
[ 18]   5.00-6.00   sec  72.1 MBytes   605 Mbits/sec
[ 20]   5.00-6.00   sec  72.0 MBytes   604 Mbits/sec
[ 22]   5.00-6.00   sec  72.0 MBytes   604 Mbits/sec
[SUM]   5.00-6.00   sec  1.11 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec   141 MBytes  1.18 Gbits/sec
[  6]   6.00-7.00   sec   141 MBytes  1.19 Gbits/sec
[  8]   6.00-7.00   sec   142 MBytes  1.19 Gbits/sec
[ 10]   6.00-7.00   sec   141 MBytes  1.18 Gbits/sec
[ 12]   6.00-7.00   sec   137 MBytes  1.15 Gbits/sec
[ 14]   6.00-7.00   sec   142 MBytes  1.19 Gbits/sec
[ 16]   6.00-7.00   sec  71.9 MBytes   603 Mbits/sec
[ 18]   6.00-7.00   sec  71.9 MBytes   603 Mbits/sec
[ 20]   6.00-7.00   sec  72.0 MBytes   604 Mbits/sec
[ 22]   6.00-7.00   sec  72.0 MBytes   604 Mbits/sec
[SUM]   6.00-7.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec   143 MBytes  1.20 Gbits/sec
[  6]   7.00-8.00   sec   141 MBytes  1.18 Gbits/sec
[  8]   7.00-8.00   sec   141 MBytes  1.18 Gbits/sec
[ 10]   7.00-8.00   sec   140 MBytes  1.17 Gbits/sec
[ 12]   7.00-8.00   sec   136 MBytes  1.14 Gbits/sec
[ 14]   7.00-8.00   sec   142 MBytes  1.19 Gbits/sec
[ 16]   7.00-8.00   sec  72.5 MBytes   608 Mbits/sec
[ 18]   7.00-8.00   sec  72.5 MBytes   608 Mbits/sec
[ 20]   7.00-8.00   sec  72.4 MBytes   607 Mbits/sec
[ 22]   7.00-8.00   sec  72.4 MBytes   607 Mbits/sec
[SUM]   7.00-8.00   sec  1.11 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec   141 MBytes  1.18 Gbits/sec
[  6]   8.00-9.00   sec   142 MBytes  1.19 Gbits/sec
[  8]   8.00-9.00   sec   142 MBytes  1.19 Gbits/sec
[ 10]   8.00-9.00   sec   141 MBytes  1.18 Gbits/sec
[ 12]   8.00-9.00   sec   139 MBytes  1.17 Gbits/sec
[ 14]   8.00-9.00   sec   141 MBytes  1.18 Gbits/sec
[ 16]   8.00-9.00   sec  71.4 MBytes   599 Mbits/sec
[ 18]   8.00-9.00   sec  71.4 MBytes   599 Mbits/sec
[ 20]   8.00-9.00   sec  71.4 MBytes   599 Mbits/sec
[ 22]   8.00-9.00   sec  71.4 MBytes   599 Mbits/sec
[SUM]   8.00-9.00   sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec   139 MBytes  1.17 Gbits/sec
[  6]   9.00-10.00  sec   142 MBytes  1.19 Gbits/sec
[  8]   9.00-10.00  sec   142 MBytes  1.19 Gbits/sec
[ 10]   9.00-10.00  sec   142 MBytes  1.19 Gbits/sec
[ 12]   9.00-10.00  sec   138 MBytes  1.16 Gbits/sec
[ 14]   9.00-10.00  sec   142 MBytes  1.19 Gbits/sec
[ 16]   9.00-10.00  sec  71.4 MBytes   599 Mbits/sec
[ 18]   9.00-10.00  sec  71.4 MBytes   599 Mbits/sec
[ 20]   9.00-10.00  sec  71.5 MBytes   600 Mbits/sec
[ 22]   9.00-10.00  sec  71.5 MBytes   600 Mbits/sec
[SUM]   9.00-10.00  sec  1.10 GBytes  9.49 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  1.37 GBytes  1.18 Gbits/sec                  sender
[  4]   0.00-10.00  sec  1.37 GBytes  1.18 Gbits/sec                  receiver
[  6]   0.00-10.00  sec  1.38 GBytes  1.18 Gbits/sec                  sender
[  6]   0.00-10.00  sec  1.38 GBytes  1.18 Gbits/sec                  receiver
[  8]   0.00-10.00  sec  1.40 GBytes  1.20 Gbits/sec                  sender
[  8]   0.00-10.00  sec  1.40 GBytes  1.20 Gbits/sec                  receiver
[ 10]   0.00-10.00  sec  1.35 GBytes  1.16 Gbits/sec                  sender
[ 10]   0.00-10.00  sec  1.35 GBytes  1.16 Gbits/sec                  receiver
[ 12]   0.00-10.00  sec  1.34 GBytes  1.15 Gbits/sec                  sender
[ 12]   0.00-10.00  sec  1.34 GBytes  1.15 Gbits/sec                  receiver
[ 14]   0.00-10.00  sec  1.38 GBytes  1.19 Gbits/sec                  sender
[ 14]   0.00-10.00  sec  1.38 GBytes  1.19 Gbits/sec                  receiver
[ 16]   0.00-10.00  sec   748 MBytes   627 Mbits/sec                  sender
[ 16]   0.00-10.00  sec   748 MBytes   627 Mbits/sec                  receiver
[ 18]   0.00-10.00  sec   677 MBytes   568 Mbits/sec                  sender
[ 18]   0.00-10.00  sec   677 MBytes   568 Mbits/sec                  receiver
[ 20]   0.00-10.00  sec   702 MBytes   589 Mbits/sec                  sender
[ 20]   0.00-10.00  sec   702 MBytes   589 Mbits/sec                  receiver
[ 22]   0.00-10.00  sec   768 MBytes   644 Mbits/sec                  sender
[ 22]   0.00-10.00  sec   767 MBytes   644 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  11.1 GBytes  9.49 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  11.0 GBytes  9.49 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -P 1
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.175 port 50479 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.01   sec   356 MBytes  2.96 Gbits/sec
[  4]   1.01-2.00   sec   667 MBytes  5.64 Gbits/sec
[  4]   2.00-3.00   sec   653 MBytes  5.47 Gbits/sec
[  4]   3.00-4.00   sec   679 MBytes  5.71 Gbits/sec
[  4]   4.00-5.00   sec   560 MBytes  4.69 Gbits/sec
[  4]   5.00-6.00   sec   632 MBytes  5.30 Gbits/sec
[  4]   6.00-7.00   sec   657 MBytes  5.50 Gbits/sec
[  4]   7.00-8.00   sec   718 MBytes  6.03 Gbits/sec
[  4]   8.00-9.00   sec   702 MBytes  5.88 Gbits/sec
[  4]   9.00-10.00  sec   699 MBytes  5.86 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  6.17 GBytes  5.30 Gbits/sec                  sender
[  4]   0.00-10.00  sec  6.17 GBytes  5.30 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -P 2
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.175 port 50482 connected to 192.168.1.33 port 5201
[  6] local 192.168.1.175 port 50483 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   665 MBytes  5.58 Gbits/sec
[  6]   0.00-1.00   sec   420 MBytes  3.52 Gbits/sec
[SUM]   0.00-1.00   sec  1.06 GBytes  9.10 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.00-2.00   sec   582 MBytes  4.88 Gbits/sec
[  6]   1.00-2.00   sec   510 MBytes  4.28 Gbits/sec
[SUM]   1.00-2.00   sec  1.07 GBytes  9.16 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec   572 MBytes  4.80 Gbits/sec
[  6]   2.00-3.00   sec   533 MBytes  4.47 Gbits/sec
[SUM]   2.00-3.00   sec  1.08 GBytes  9.27 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec   553 MBytes  4.64 Gbits/sec
[  6]   3.00-4.00   sec   554 MBytes  4.64 Gbits/sec
[SUM]   3.00-4.00   sec  1.08 GBytes  9.28 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec   560 MBytes  4.70 Gbits/sec
[  6]   4.00-5.00   sec   550 MBytes  4.61 Gbits/sec
[SUM]   4.00-5.00   sec  1.08 GBytes  9.31 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec   570 MBytes  4.78 Gbits/sec
[  6]   5.00-6.00   sec   542 MBytes  4.55 Gbits/sec
[SUM]   5.00-6.00   sec  1.09 GBytes  9.33 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec   563 MBytes  4.72 Gbits/sec
[  6]   6.00-7.00   sec   545 MBytes  4.57 Gbits/sec
[SUM]   6.00-7.00   sec  1.08 GBytes  9.29 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec   552 MBytes  4.63 Gbits/sec
[  6]   7.00-8.00   sec   556 MBytes  4.66 Gbits/sec
[SUM]   7.00-8.00   sec  1.08 GBytes  9.29 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec   564 MBytes  4.73 Gbits/sec
[  6]   8.00-9.00   sec   547 MBytes  4.59 Gbits/sec
[SUM]   8.00-9.00   sec  1.08 GBytes  9.32 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec   561 MBytes  4.70 Gbits/sec
[  6]   9.00-10.00  sec   547 MBytes  4.59 Gbits/sec
[SUM]   9.00-10.00  sec  1.08 GBytes  9.29 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  5.61 GBytes  4.82 Gbits/sec                  sender
[  4]   0.00-10.00  sec  5.61 GBytes  4.82 Gbits/sec                  receiver
[  6]   0.00-10.00  sec  5.18 GBytes  4.45 Gbits/sec                  sender
[  6]   0.00-10.00  sec  5.18 GBytes  4.45 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  10.8 GBytes  9.26 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  10.8 GBytes  9.26 Gbits/sec                  receiver

iperf Done.
PS C:\Users\Matthias\Desktop\iperf-3.1.3-win64> .\iperf3.exe iperf3 -c 192.168.1.33 -p 5201 -P 3
Connecting to host 192.168.1.33, port 5201
[  4] local 192.168.1.175 port 50487 connected to 192.168.1.33 port 5201
[  6] local 192.168.1.175 port 50488 connected to 192.168.1.33 port 5201
[  8] local 192.168.1.175 port 50489 connected to 192.168.1.33 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   377 MBytes  3.16 Gbits/sec
[  6]   0.00-1.00   sec   379 MBytes  3.18 Gbits/sec
[  8]   0.00-1.00   sec   373 MBytes  3.13 Gbits/sec
[SUM]   0.00-1.00   sec  1.10 GBytes  9.47 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   1.00-2.00   sec   382 MBytes  3.21 Gbits/sec
[  6]   1.00-2.00   sec   382 MBytes  3.21 Gbits/sec
[  8]   1.00-2.00   sec   364 MBytes  3.05 Gbits/sec
[SUM]   1.00-2.00   sec  1.10 GBytes  9.47 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   2.00-3.00   sec   382 MBytes  3.20 Gbits/sec
[  6]   2.00-3.00   sec   384 MBytes  3.22 Gbits/sec
[  8]   2.00-3.00   sec   364 MBytes  3.05 Gbits/sec
[SUM]   2.00-3.00   sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   3.00-4.00   sec   373 MBytes  3.13 Gbits/sec
[  6]   3.00-4.00   sec   384 MBytes  3.23 Gbits/sec
[  8]   3.00-4.00   sec   372 MBytes  3.12 Gbits/sec
[SUM]   3.00-4.00   sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   4.00-5.00   sec   381 MBytes  3.20 Gbits/sec
[  6]   4.00-5.00   sec   383 MBytes  3.21 Gbits/sec
[  8]   4.00-5.00   sec   366 MBytes  3.07 Gbits/sec
[SUM]   4.00-5.00   sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   5.00-6.00   sec   378 MBytes  3.17 Gbits/sec
[  6]   5.00-6.00   sec   377 MBytes  3.17 Gbits/sec
[  8]   5.00-6.00   sec   375 MBytes  3.15 Gbits/sec
[SUM]   5.00-6.00   sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   6.00-7.00   sec   382 MBytes  3.20 Gbits/sec
[  6]   6.00-7.00   sec   377 MBytes  3.16 Gbits/sec
[  8]   6.00-7.00   sec   371 MBytes  3.11 Gbits/sec
[SUM]   6.00-7.00   sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   7.00-8.00   sec   373 MBytes  3.13 Gbits/sec
[  6]   7.00-8.00   sec   387 MBytes  3.25 Gbits/sec
[  8]   7.00-8.00   sec   360 MBytes  3.02 Gbits/sec
[SUM]   7.00-8.00   sec  1.09 GBytes  9.39 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   8.00-9.00   sec   372 MBytes  3.12 Gbits/sec
[  6]   8.00-9.00   sec   376 MBytes  3.16 Gbits/sec
[  8]   8.00-9.00   sec   354 MBytes  2.97 Gbits/sec
[SUM]   8.00-9.00   sec  1.08 GBytes  9.25 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  4]   9.00-10.00  sec   378 MBytes  3.17 Gbits/sec
[  6]   9.00-10.00  sec   380 MBytes  3.18 Gbits/sec
[  8]   9.00-10.00  sec   373 MBytes  3.13 Gbits/sec
[SUM]   9.00-10.00  sec  1.10 GBytes  9.48 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  3.69 GBytes  3.17 Gbits/sec                  sender
[  4]   0.00-10.00  sec  3.69 GBytes  3.17 Gbits/sec                  receiver
[  6]   0.00-10.00  sec  3.72 GBytes  3.20 Gbits/sec                  sender
[  6]   0.00-10.00  sec  3.72 GBytes  3.20 Gbits/sec                  receiver
[  8]   0.00-10.00  sec  3.59 GBytes  3.08 Gbits/sec                  sender
[  8]   0.00-10.00  sec  3.59 GBytes  3.08 Gbits/sec                  receiver
[SUM]   0.00-10.00  sec  11.0 GBytes  9.45 Gbits/sec                  sender
[SUM]   0.00-10.00  sec  11.0 GBytes  9.45 Gbits/sec                  receiver

iperf Done.


Windows screenshot of the taskmanager with the X520 interface confirms:

note: added the following to iperf command: '-P 1', '-P 2', '-P 3'

1678392069026.png


So 10 Gbe confirmed working through the switch.

For the curious ones, i got to ' -P 30' till results got worse (due to cpu limits i figure, single threaded program?).
 
You must log in or register to reply here.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "10Gbe woes - and upgrade time!"

Similar threads

R
  • Locked
intel x520-da2 SFP+ 10Gb FreeNAS-9.3
Replies
3
Views
3K
shubert
S
R
  • Locked
Networking SNMP
Replies
5
Views
3K
tvsjr
T
L
How does jail NAT port forwarding work?
Replies
1
Views
4K
LMCDZ
L
R
  • Locked
intel x520-da2, with SFP+ 10Gb. FreeNAS-9.3
Replies
7
Views
4K
jgreco
jgreco
P
  • Locked
slow transfer rates
Replies
13
Views
4K
Psychoteur
P
Top