turborierer
Cadet
- Joined
- Dec 2, 2022
- Messages
- 3
Dear all,
the problem: TrueNAS Scale Sever with SuperMico MB was running good but the network speed was to low with 1 Gbit ethernet. So I put in a 10 Gbit PCI card, connected the server via fiber optic cable to the switch which is also indicating a 10 gibt connection. But if I run a iperf3 server on TrueNAS I just get 1 Gbit. Also the VM running on this server is showing that the "br0" where its attached only got 1 Gbit.
So I did some research and I think its related to the initial NIC which is somehow stick to the old network interface or includes it and holding the NIC back from using the greater bandwidth of the new card. Some redundancy setting? Anyway I'm lost to tell TrueNAS how to only use the new network card with full bandwidth.
Any help would be highly appreciated how to stick enp1s0f0 to br0 and tell them to use the full 10 Gbit.
Here come some prints about my setup:
the problem: TrueNAS Scale Sever with SuperMico MB was running good but the network speed was to low with 1 Gbit ethernet. So I put in a 10 Gbit PCI card, connected the server via fiber optic cable to the switch which is also indicating a 10 gibt connection. But if I run a iperf3 server on TrueNAS I just get 1 Gbit. Also the VM running on this server is showing that the "br0" where its attached only got 1 Gbit.
So I did some research and I think its related to the initial NIC which is somehow stick to the old network interface or includes it and holding the NIC back from using the greater bandwidth of the new card. Some redundancy setting? Anyway I'm lost to tell TrueNAS how to only use the new network card with full bandwidth.
Any help would be highly appreciated how to stick enp1s0f0 to br0 and tell them to use the full 10 Gbit.
Here come some prints about my setup:
Code:
lspci | egrep -i --color 'network|ethernet' 01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 06:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection 07:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
Code:
sudo lshw -class network -short H/W path Device Class Description ============================================================== /0/100/1/0 enp1s0f0 network 82599ES 10-Gigabit SFI/SFP+ Network Connection /0/100/1/0.1 enp1s0f1 network 82599ES 10-Gigabit SFI/SFP+ Network Connection /0/100/1c.4/0 enp6s0 network 82574L Gigabit Network Connection /0/100/1c.5/0 enp7s0 network 82574L Gigabit Network Connection /1 br0 network Ethernet interface /2 vnet0 network Ethernet interface /3 kube-bridge network Ethernet interface /4 kube-dummy-if network Ethernet interface /5 vethade24a32 network Ethernet interface /6 veth99e65218 network Ethernet interface
Code:
ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp6s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 00:25:90:30:50:14 brd ff:ff:ff:ff:ff:ff 3: enp7s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 00:25:90:30:50:15 brd ff:ff:ff:ff:ff:ff 4: enp1s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000 link/ether 00:11:0a:68:b5:64 brd ff:ff:ff:ff:ff:ff 5: enp1s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP mode DEFAULT group default qlen 1000 link/ether 00:11:0a:68:b5:65 brd ff:ff:ff:ff:ff:ff 6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether 2a:5d:d5:66:f2:4c brd ff:ff:ff:ff:ff:ff 7: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue master br0 state UNKNOWN mode DEFAULT group default qlen 1000 link/ether fe:a0:98:3f:19:d9 brd ff:ff:ff:ff:ff:ff 8: kube-bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether aa:14:80:2f:61:b9 brd ff:ff:ff:ff:ff:ff 9: kube-dummy-if: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default link/ether 5e:c8:b7:c4:d5:c5 brd ff:ff:ff:ff:ff:ff 10: vethade24a32@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master kube-bridge state UP mode DEFAULT group default link/ether ea:9f:bf:53:82:85 brd ff:ff:ff:ff:ff:ff link-netnsid 0 11: veth99e65218@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master kube-bridge state UP mode DEFAULT group default link/ether 5e:4c:b2:a4:d6:b6 brd ff:ff:ff:ff:ff:ff link-netnsid 1
Code:
ip -br -c link show lo UNKNOWN 00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP> enp6s0 DOWN 00:25:90:30:50:14 <NO-CARRIER,BROADCAST,MULTICAST,UP> enp7s0 DOWN 00:25:90:30:50:15 <NO-CARRIER,BROADCAST,MULTICAST,UP> enp1s0f0 DOWN 00:11:0a:68:b5:64 <BROADCAST,MULTICAST> enp1s0f1 UP 00:11:0a:68:b5:65 <BROADCAST,MULTICAST,UP,LOWER_UP> br0 UP 2a:5d:d5:66:f2:4c <BROADCAST,MULTICAST,UP,LOWER_UP> vnet0 UNKNOWN fe:a0:98:3f:19:d9 <BROADCAST,MULTICAST,UP,LOWER_UP> kube-bridge UP aa:14:80:2f:61:b9 <BROADCAST,MULTICAST,UP,LOWER_UP> kube-dummy-if UNKNOWN 5e:c8:b7:c4:d5:c5 <BROADCAST,NOARP,UP,LOWER_UP> vethade24a32@if3 UP ea:9f:bf:53:82:85 <BROADCAST,MULTICAST,UP,LOWER_UP> veth99e65218@if3 UP 5e:4c:b2:a4:d6:b6 <BROADCAST,MULTICAST,UP,LOWER_UP>
Code:
lshw -class network *-network:0 DISABLED description: Ethernet interface product: 82599ES 10-Gigabit SFI/SFP+ Network Connection vendor: Intel Corporation physical id: 0 bus info: pci@0000:01:00.0 logical name: enp1s0f0 version: 01 serial: 00:11:0a:68:b5:64 capacity: 10Gbit/s width: 32 bits clock: 33MHz capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical fibre 10000bt-fd configuration: autonegotiation=off broadcast=yes driver=ixgbe driverversion=5.10.131+truenas firmware=0x80000835, 1.2836.0 latency=0 link=no multicast=yes port=fibre resources: irq:33 memory:faa00000-faafffff ioport:ac00(size=32) memory:fa97c000-fa97ffff memory:fa980000-fa9fffff memory:c0000000-c00fffff memory:c0100000-c01fffff *-network:1 description: Ethernet interface product: 82599ES 10-Gigabit SFI/SFP+ Network Connection vendor: Intel Corporation physical id: 0.1 bus info: pci@0000:01:00.1 logical name: enp1s0f1 version: 01 serial: 00:11:0a:68:b5:65 size: 10Gbit/s capacity: 10Gbit/s width: 32 bits clock: 33MHz capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical fibre 10000bt-fd configuration: autonegotiation=off broadcast=yes driver=ixgbe driverversion=5.10.131+truenas duplex=full firmware=0x80000835, 1.2836.0 latency=0 link=yes multicast=yes port=fibre speed=10Gbit/s resources: irq:79 memory:fa800000-fa8fffff ioport:a880(size=32) memory:fa77c000-fa77ffff memory:fa780000-fa7fffff memory:c0200000-c02fffff memory:c0300000-c03fffff *-network description: Ethernet interface product: 82574L Gigabit Network Connection vendor: Intel Corporation physical id: 0 bus info: pci@0000:06:00.0 logical name: enp6s0 version: 00 serial: 00:25:90:30:50:14 capacity: 1Gbit/s width: 32 bits clock: 33MHz capabilities: pm msi pciexpress msix bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=5.10.131+truenas firmware=1.8-0 latency=0 link=no multicast=yes port=twisted pair resources: irq:16 memory:fbce0000-fbcfffff ioport:dc00(size=32) memory:fbcdc000-fbcdffff *-network description: Ethernet interface product: 82574L Gigabit Network Connection vendor: Intel Corporation physical id: 0 bus info: pci@0000:07:00.0 logical name: enp7s0 version: 00 serial: 00:25:90:30:50:15 capacity: 1Gbit/s width: 32 bits clock: 33MHz capabilities: pm msi pciexpress msix bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation configuration: autonegotiation=on broadcast=yes driver=e1000e driverversion=5.10.131+truenas firmware=1.8-0 latency=0 link=no multicast=yes port=twisted pair resources: irq:17 memory:fbde0000-fbdfffff ioport:ec00(size=32) memory:fbddc000-fbddffff *-network:0 description: Ethernet interface physical id: 1 logical name: br0 serial: 2a:5d:d5:66:f2:4c size: 10Gbit/s capabilities: ethernet physical configuration: autonegotiation=off broadcast=yes driver=bridge driverversion=2.3 firmware=N/A ip=192.168.1.31 link=yes multicast=yes speed=10Gbit/s *-network:1 description: Ethernet interface physical id: 2 logical name: vnet0 serial: fe:a0:98:3f:19:d9 size: 10Mbit/s capabilities: ethernet physical configuration: autonegotiation=off broadcast=yes driver=tun driverversion=1.6 duplex=full link=yes multicast=yes port=twisted pair speed=10Mbit/s *-network:2 description: Ethernet interface physical id: 3 logical name: kube-bridge serial: aa:14:80:2f:61:b9 size: 10Gbit/s capabilities: ethernet physical configuration: autonegotiation=off broadcast=yes driver=bridge driverversion=2.3 firmware=N/A ip=172.16.0.1 link=yes multicast=yes speed=10Gbit/s *-network:3 description: Ethernet interface physical id: 4 logical name: kube-dummy-if serial: 5e:c8:b7:c4:d5:c5 capabilities: ethernet physical configuration: broadcast=yes driver=dummy driverversion=5.10.131+truenas ip=172.17.0.1 *-network:4 description: Ethernet interface physical id: 5 logical name: vethade24a32 serial: ea:9f:bf:53:82:85 size: 10Gbit/s capabilities: ethernet physical configuration: autonegotiation=off broadcast=yes driver=veth driverversion=1.0 duplex=full link=yes multicast=yes port=twisted pair speed=10Gbit/s *-network:5 description: Ethernet interface physical id: 6 logical name: veth99e65218 serial: 5e:4c:b2:a4:d6:b6 size: 10Gbit/s capabilities: ethernet physical configuration: autonegotiation=off broadcast=yes driver=veth driverversion=1.0 duplex=full link=yes multicast=yes port=twisted pair speed=10Gbit/s