milizhang
Cadet
- Joined
- Nov 29, 2018
- Messages
- 2
Server Hardware: A1SAM-2550F (Atom C2550), 32GB RAM, 4x OnChip Intel GbE NIC, with Link Aggregation (LACP).
Network Hardware: Unifi GbE Switches and 11ac Wifi APs (AC-AP-Pro).
The machine is serving both as a iSCSI Device for a VM Pool, as well as a personal file server.
It gets generally good performance on wired access, and generally acceptable Wifi performance on several machines. However, when accessing from my MacBook Pro (MacOS 10.14.1, Broadcom BCM43xx series Wifi), TCP connections could get extremely unstable.
iPerf3 tests shows that in many cases that packet transmits OK in the first second or two, then it pretty much stops. A typical pattern looks like below:
I applied packet capture and trying to understand what is happening, and what I figured out is that at some point the server is sending hundreds of DupACKs, and then pretty much stop responding to following packets until retransmission happened seconds later.
Interestingly, it seems like turning off delayed_ack in kernel does prevent the issue from happening - however it looks like doing so will cost more CPU resources with network is loaded, and computation power is quite constrained on Atom based systems.
I am not sure whether this is a FreeBSD kernel bug or somethings is wrong with my network setup. However it seems like this Macbook Pro have no problem iperf3 to other Linux servers through the same network equipments, and other Wifi devices such as a Windows tablet can hold a very stable connection with the FreeNAS server from the same AP.
A pcap file, captured from client can be found at https://www.dropbox.com/s/jet82s4jbeig5cl/Laptop.pcapng.zip. (I had another capture on the server, but it shows the same pattern)
I would really appreciate if anyone can help me to solve the mystery. Thanks.
Network Hardware: Unifi GbE Switches and 11ac Wifi APs (AC-AP-Pro).
The machine is serving both as a iSCSI Device for a VM Pool, as well as a personal file server.
It gets generally good performance on wired access, and generally acceptable Wifi performance on several machines. However, when accessing from my MacBook Pro (MacOS 10.14.1, Broadcom BCM43xx series Wifi), TCP connections could get extremely unstable.
iPerf3 tests shows that in many cases that packet transmits OK in the first second or two, then it pretty much stops. A typical pattern looks like below:
Code:
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 10 second test, tos 0 [ ID] Interval Transfer Bitrate [ 7] 0.00-1.00 sec 4.51 MBytes 37.8 Mbits/sec [ 7] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec [ 7] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec [ 7] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec [ 7] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec [ 7] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec [ 7] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec [ 7] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec [ 7] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec [ 7] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec
I applied packet capture and trying to understand what is happening, and what I figured out is that at some point the server is sending hundreds of DupACKs, and then pretty much stop responding to following packets until retransmission happened seconds later.
Interestingly, it seems like turning off delayed_ack in kernel does prevent the issue from happening - however it looks like doing so will cost more CPU resources with network is loaded, and computation power is quite constrained on Atom based systems.
I am not sure whether this is a FreeBSD kernel bug or somethings is wrong with my network setup. However it seems like this Macbook Pro have no problem iperf3 to other Linux servers through the same network equipments, and other Wifi devices such as a Windows tablet can hold a very stable connection with the FreeNAS server from the same AP.
A pcap file, captured from client can be found at https://www.dropbox.com/s/jet82s4jbeig5cl/Laptop.pcapng.zip. (I had another capture on the server, but it shows the same pattern)
I would really appreciate if anyone can help me to solve the mystery. Thanks.