NickF
Guru
- Joined
- Jun 12, 2014
- Messages
- 763
I opened a ticket https://ixsystems.atlassian.net/browse/NAS-119508
Is anyone else seeing behavior like this?
This is a weird one. In terms of change state I’m not sure when this stopped functioning properly, but I did two things recently:
The bond is functioning as expected and properly negotiating LACPPDUs
It is reporting a 20 Gigabit connection:
On SCALE in the GUI I can see the bond reports 20Gb:
In the CLI, Ethrool reports 10 Gigabit as well:
But when I do an ip link show I am seeing a qlen of 1000 on the bond:
When I attempt to do a file transfer from a computer, they seem to be limited to about 400mb, whereas before I was hitting much higher. confirming this, I ran an iperf test between the two:
Is anyone else seeing behavior like this?
This is a weird one. In terms of change state I’m not sure when this stopped functioning properly, but I did two things recently:
- I made a bridge with no NICs attached
- I upgraded from SCALE RC1 to 22.12.0
The bond is functioning as expected and properly negotiating LACPPDUs
Code:
Channel group 11 neighbors LACP port Admin Oper Port Port Port Flags Priority Dev ID Age key Key Number State Te1/1/1 FA 255 3cfd.fed7.bfcc 20s 0x0 0xF 0x1 0x3F Te1/1/2 FA 255 3cfd.fed7.bfcc 16s 0x0 0xF 0x2 0x3F
It is reporting a 20 Gigabit connection:
Code:
#show interfaces port-channel 11 Port-channel11 is up, line protocol is up (connected) Hardware is EtherChannel, address is 6c6c.d3c7.54b5 (bia 6c6c.d3c7.54b5) Description: scale_prod_uplink MTU 1500 bytes, BW 20000000 Kbit/sec, DLY 10 usec, reliability 255/255, txload 1/255, rxload 1/255 Encapsulation ARPA, loopback not set Keepalive set (10 sec) Full-duplex, 10Gb/s, link type is auto, media type is N/A input flow-control is on, output flow-control is unsupported Members in this channel: Te1/1/1 Te1/1/2 ARP type: ARPA, ARP Timeout 04:00:00 Last input never, output 00:00:01, output hang never Last clearing of "show interface" counters never Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 1986779 Queueing strategy: fifo Output queue: 0/40 (size/max) 5 minute input rate 43000 bits/sec, 22 packets/sec 5 minute output rate 125000 bits/sec, 48 packets/sec 132187482412 packets input, 188018602975625 bytes, 0 no buffer Received 11549916 broadcasts (7259457 multicasts) 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 watchdog, 7259457 multicast, 0 pause input 0 input packets with dribble condition detected 119602936430 packets output, 168834842361229 bytes, 0 underruns Output 16934790 broadcasts (0 multicasts) 0 output errors, 0 collisions, 5 interface resets 0 unknown protocol drops 0 babbles, 0 late collision, 0 deferred 0 lost carrier, 0 no carrier, 0 pause output 0 output buffer failures, 0 output buffers swapped out
On SCALE in the GUI I can see the bond reports 20Gb:
In the CLI, Ethrool reports 10 Gigabit as well:
Code:
root@prod[~]# ethtool enp68s0f0 Settings for enp68s0f0: Supported ports: [ FIBRE ] Supported link modes: 10000baseT/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: No Supported FEC modes: Not reported Advertised link modes: 10000baseT/Full Advertised pause frame use: No Advertised auto-negotiation: No Advertised FEC modes: Not reported Speed: 10000Mb/s Duplex: Full Auto-negotiation: off Port: Direct Attach Copper PHYAD: 0 Transceiver: internal Supports Wake-on: d Wake-on: d Current message level: 0x00000007 (7) drv probe link Link detected: yes
Code:
root@prod[~]# ethtool enp68s0f1 Settings for enp68s0f1: Supported ports: [ FIBRE ] Supported link modes: 10000baseT/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: No Supported FEC modes: Not reported Advertised link modes: 10000baseT/Full Advertised pause frame use: No Advertised auto-negotiation: No Advertised FEC modes: Not reported Speed: 10000Mb/s Duplex: Full Auto-negotiation: off Port: Direct Attach Copper PHYAD: 0 Transceiver: internal Supports Wake-on: d Wake-on: d Current message level: 0x00000007 (7) drv probe link Link detected: yes
But when I do an ip link show I am seeing a qlen of 1000 on the bond:
Code:
root@prod[~]# ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp68s0f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master b ond0 state UP mode DEFAULT group default qlen 1000 link/ether 3c:fd:fe:d7:bf:cc brd ff:ff:ff:ff:ff:ff 3: enp68s0f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master b ond0 state UP mode DEFAULT group default qlen 1000 link/ether 3c:fd:fe:d7:bf:cc brd ff:ff:ff:ff:ff:ff permaddr 3c:fd:fe:d7:bf:c d 4: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 2e:4e:3f:19:e1:21 brd ff:ff:ff:ff:ff:ff 5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether 3c:fd:fe:d7:bf:cc brd ff:ff:ff:ff:ff:ff 6: vlan60@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether 3c:fd:fe:d7:bf:cc brd ff:ff:ff:ff:ff:ff 7: vlan10@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether 3c:fd:fe:d7:bf:cc brd ff:ff:ff:ff:ff:ff 8: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode D EFAULT group default qlen 1000 link/ether ca:8b:a7:75:bc:cd brd ff:ff:ff:ff:ff:ff 9: macvtap0@vlan60: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue sta te UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:22:12:da brd ff:ff:ff:ff:ff:ff 10: macvtap1@vlan60: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue st ate UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:43:be:2a brd ff:ff:ff:ff:ff:ff 11: macvtap2@vlan60: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue st ate UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:5e:d3:8f brd ff:ff:ff:ff:ff:ff 12: macvtap3@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue sta te UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:1f:b2:c5 brd ff:ff:ff:ff:ff:ff 13: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 s tate UNKNOWN mode DEFAULT group default qlen 1000 link/ether fe:a0:98:01:94:59 brd ff:ff:ff:ff:ff:ff 14: macvtap4@vlan10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue st ate UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:38:46:98 brd ff:ff:ff:ff:ff:ff 15: macvtap5@vlan60: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue st ate UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:62:0d:b6 brd ff:ff:ff:ff:ff:ff 16: macvtap6@vlan60: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue st ate UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:35:38:0f brd ff:ff:ff:ff:ff:ff 17: macvtap7@vlan60: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue st ate UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:26:74:f9 brd ff:ff:ff:ff:ff:ff 18: macvtap8@vlan10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue st ate UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:43:54:3b brd ff:ff:ff:ff:ff:ff 19: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 s tate UNKNOWN mode DEFAULT group default qlen 1000 link/ether fe:a0:98:4a:1a:f7 brd ff:ff:ff:ff:ff:ff 20: macvtap9@vlan60: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue st ate UP mode DEFAULT group default qlen 500 link/ether 00:a0:98:13:0f:4f brd ff:ff:ff:ff:ff:ff 21: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 s tate UNKNOWN mode DEFAULT group default qlen 1000 link/ether fe:a0:98:6b:0a:70 brd ff:ff:ff:ff:ff:ff
When I attempt to do a file transfer from a computer, they seem to be limited to about 400mb, whereas before I was hitting much higher. confirming this, I ran an iperf test between the two:
Code:
C:\Users\nickf\Downloads\iperf-3.1.3-win64>iperf3 -c prod.fusco.me Connecting to host prod.fusco.me, port 5201 [ 4] local 10.69.10.67 port 44812 connected to 10.69.40.8 port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 47.9 MBytes 401 Mbits/sec [ 4] 1.00-2.00 sec 47.5 MBytes 399 Mbits/sec [ 4] 2.00-3.00 sec 48.0 MBytes 402 Mbits/sec [ 4] 3.00-4.00 sec 47.8 MBytes 401 Mbits/sec [ 4] 4.00-5.00 sec 47.2 MBytes 396 Mbits/sec [ 4] 5.00-6.00 sec 48.0 MBytes 403 Mbits/sec [ 4] 6.00-7.00 sec 47.1 MBytes 395 Mbits/sec [ 4] 7.00-8.00 sec 47.9 MBytes 401 Mbits/sec [ 4] 8.00-9.00 sec 47.8 MBytes 400 Mbits/sec [ 4] 9.00-10.00 sec 47.5 MBytes 399 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 477 MBytes 400 Mbits/sec sender [ 4] 0.00-10.00 sec 476 MBytes 400 Mbits/sec