Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

Can't write properly from Mac OSX VM to Freenas

borisko

Neophyte
Joined
Jun 27, 2019
Messages
9
Hi everyone,

I am at an impass. After several iperf3 tests and Blackmagic write/reads, I realized all connections work fine from all the computers at home, except uploads/writes from my Macintosh VM (under Proxmox server) to the NAS (freenas).

I posted on the Proxmox forums as well, but I figured the community here might shed light, and it could be a config I oversaw somewhere on the nas itself.

As you will see below, I've ran tests from all points (Proxmox node itself to and from several machines including over wifi and ethernet, machines to and from the NAS, etc) and the only thing that doesn't work is from my Mac VM to the NAS.

It feels like the data stream gets saturated when I do so. It starts writing/sending, and everything related to internet begins to slow down: Freenas GUI unresponsive in my mac VM, write stops, etc. You can see through the iperf tests that a first packet is sent, then 0, 0, 0... etc until the end of the test.

I have not set jumbo frames anywhere, tried activating NAT acceleration on my router (ASUS Ac-88u, which is supposed to be a very capable router) and ethernet cable works fine as obviously showing with the proxmox server read/write tests.

Is vmxnet3 or any setup from my Proxmox machine the bottleneck? Could there be a setting I missed, or a conflict between Proxmox routing the data and the router receiving it? I am pretty sure I have a double NAT setup, but since there isn't any issue with other machines except my VM, that seems unlikely.

Last thing is that I have set up Syncthing bterween the Mac VM and the NAS a few days ago. I am unsure I had those problems before setting that up (it's a brand new build for both) but uninstalling the whole syncthing dependencies on my mac VM and turning off the jail (then ultimately deleting it) in freenas did nothing, so I have a feeling the issue is my proxmox setup (correct me if I'm wrong of course! I'm a noob on both and here to learn).


My machines for this test:
NAS = Freenas separate machine (LAN connection, 192.168.1.45) (hardware would seem irrelevant at this point, but it is by no means recommended hardware so here it is: AMD A8-7600 | MOBO: C.A68HM-K PLUS colorful plus V18 with the Realtek RTL8111/8168/8411)
Server = Proxmox machine (Proxmox 5.4, LAN connection, 192.168.1.191)
MacOSvm = My Mac VM (Mojave 10.14.4, VMXNet3 ethernet, 192.168.1.229)
iMacWifi = authentic iMac (Mojave 10.14.5, over wifi 5ghz, 192.168.1.6)

Router: brand new Asus AC88u acting as the router and switch (I have a huawei modem to delivery main WAN, supposedly bridged)


Feel free to let me know if I need to run other tests or output other settings to investigate! I haven't setup any kind of interface, vlan or anything on Freenas so far.


Here is my interfaces config on proxmox running cat /etc/network/interfaces :

Code:
auto lo

iface lo inet loopback

iface eno2 inet manual

auto vmbr0
iface vmbr0 inet static
address 192.168.1.191
netmask 255.255.255.0
gateway 192.168.1.1
bridge_ports eno2
bridge_stp off
bridge_fd 0



Here are the iperf3 tests:

NAS >> Server

Code:
[ 5] local 192.168.1.45 port 62731 connected to 192.168.1.191 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 84.9 MBytes 712 Mbits/sec 909 598 KBytes
[ 5] 1.00-2.00 sec 83.4 MBytes 700 Mbits/sec 0 713 KBytes
[ 5] 2.00-3.00 sec 83.5 MBytes 700 Mbits/sec 0 813 KBytes
[ 5] 3.00-4.00 sec 83.4 MBytes 700 Mbits/sec 4 483 KBytes
[ 5] 4.00-5.00 sec 83.4 MBytes 700 Mbits/sec 0 622 KBytes
[ 5] 5.00-6.00 sec 83.5 MBytes 701 Mbits/sec 0 734 KBytes
[ 5] 6.00-7.00 sec 83.5 MBytes 700 Mbits/sec 0 827 KBytes
[ 5] 7.00-8.00 sec 83.4 MBytes 700 Mbits/sec 0 843 KBytes
[ 5] 8.00-9.00 sec 83.5 MBytes 700 Mbits/sec 0 858 KBytes
[ 5] 9.00-10.00 sec 83.5 MBytes 701 Mbits/sec 0 872 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 836 MBytes 701 Mbits/sec 913 sender
[ 5] 0.00-10.00 sec 835 MBytes 701 Mbits/sec receiver

iperf Done.


Server >> NAS

Code:
Accepted connection from 192.168.1.191, port 39518
[ 5] local 192.168.1.45 port 5201 connected to 192.168.1.191 port 39520
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 111 MBytes 934 Mbits/sec
[ 5] 1.00-2.00 sec 112 MBytes 937 Mbits/sec
[ 5] 2.00-3.00 sec 112 MBytes 937 Mbits/sec
[ 5] 3.00-4.00 sec 112 MBytes 936 Mbits/sec
[ 5] 4.00-5.00 sec 112 MBytes 937 Mbits/sec
[ 5] 5.00-6.00 sec 112 MBytes 937 Mbits/sec
[ 5] 6.00-7.00 sec 112 MBytes 937 Mbits/sec
[ 5] 7.00-8.00 sec 112 MBytes 937 Mbits/sec
[ 5] 8.00-9.00 sec 112 MBytes 936 Mbits/sec
[ 5] 9.00-10.00 sec 112 MBytes 937 Mbits/sec
[ 5] 10.00-10.00 sec 250 KBytes 933 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.00 sec 1.09 GBytes 936 Mbits/sec receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------


Server >> MacOSvm

Code:
Accepted connection from 192.168.1.191, port 36556
[ 5] local 192.168.1.229 port 5201 connected to 192.168.1.191 port 36558
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 158 MBytes 1.33 Gbits/sec
[ 5] 1.00-2.00 sec 178 MBytes 1.49 Gbits/sec
[ 5] 2.00-3.00 sec 177 MBytes 1.49 Gbits/sec
[ 5] 3.00-4.00 sec 177 MBytes 1.49 Gbits/sec
[ 5] 4.00-5.00 sec 178 MBytes 1.49 Gbits/sec
[ 5] 5.00-6.00 sec 176 MBytes 1.48 Gbits/sec
[ 5] 6.00-7.00 sec 178 MBytes 1.49 Gbits/sec
[ 5] 7.00-8.00 sec 177 MBytes 1.48 Gbits/sec
[ 5] 8.00-9.00 sec 178 MBytes 1.49 Gbits/sec
[ 5] 9.00-10.00 sec 177 MBytes 1.48 Gbits/sec
[ 5] 10.00-10.05 sec 8.33 MBytes 1.51 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.05 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.05 sec 1.72 GBytes 1.47 Gbits/sec


MacOSvm >> Server

Code:
Accepted connection from 192.168.1.229, port 64307
[ 5] local 192.168.1.191 port 5201 connected to 192.168.1.229 port 64308
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 195 MBytes 1.64 Gbits/sec
[ 5] 1.00-2.00 sec 874 MBytes 7.34 Gbits/sec
[ 5] 2.00-3.00 sec 455 MBytes 3.82 Gbits/sec
[ 5] 3.00-4.00 sec 204 MBytes 1.71 Gbits/sec
[ 5] 4.00-5.00 sec 977 MBytes 8.20 Gbits/sec
[ 5] 5.00-6.00 sec 867 MBytes 7.27 Gbits/sec
[ 5] 6.00-7.00 sec 1.16 GBytes 10.0 Gbits/sec
[ 5] 7.00-8.00 sec 220 MBytes 1.84 Gbits/sec
[ 5] 8.00-9.00 sec 818 MBytes 6.86 Gbits/sec
[ 5] 9.00-10.00 sec 177 MBytes 1.48 Gbits/sec
[ 5] 10.00-10.00 sec 0.00 Bytes 0.00 bits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.00 sec 5.84 GBytes 5.02 Gbits/sec receiver


MacOSvm >> NAS

Code:
Accepted connection from 192.168.1.229, port 64418
[ 5] local 192.168.1.45 port 5201 connected to 192.168.1.229 port 64419
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.01 sec 56.4 MBytes 469 Mbits/sec
[ 5] 1.01-2.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 2.06-3.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 3.06-4.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 4.06-5.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 5.06-6.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 6.06-7.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 7.06-8.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 8.06-9.05 sec 0.00 Bytes 0.00 bits/sec
[ 5] 9.05-10.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 10.06-11.01 sec 0.00 Bytes 0.00 bits/sec
[ 5] 11.01-12.05 sec 0.00 Bytes 0.00 bits/sec
[ 5] 12.05-13.06 sec 0.00 Bytes 0.00 bits/sec
[ 5] 13.06-13.53 sec 0.00 Bytes 0.00 bits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-13.53 sec 56.4 MBytes 35.0 Mbits/sec receiver


that's where it craps the bed...

NAS >> MacOSvm

Code:
Accepted connection from 192.168.1.45, port 24734
[ 5] local 192.168.1.229 port 5201 connected to 192.168.1.45 port 45358
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 55.8 MBytes 468 Mbits/sec
[ 5] 1.00-2.00 sec 83.7 MBytes 702 Mbits/sec
[ 5] 2.00-3.00 sec 83.6 MBytes 702 Mbits/sec
[ 5] 3.00-4.00 sec 83.6 MBytes 701 Mbits/sec
[ 5] 4.00-5.00 sec 83.6 MBytes 701 Mbits/sec
[ 5] 5.00-6.00 sec 83.5 MBytes 701 Mbits/sec
[ 5] 6.00-7.00 sec 83.7 MBytes 702 Mbits/sec
[ 5] 7.00-8.00 sec 83.7 MBytes 702 Mbits/sec
[ 5] 8.00-9.00 sec 83.5 MBytes 701 Mbits/sec
[ 5] 9.00-10.00 sec 83.7 MBytes 702 Mbits/sec
[ 5] 10.00-10.15 sec 12.9 MBytes 701 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.15 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.15 sec 821 MBytes 679 Mbits/sec receiver


all fine the other way !!!


iMacWifi >> MacOSvm

Code:
[ 4] local 192.168.1.229 port 65136 connected to 192.168.1.132 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 87.8 MBytes 736 Mbits/sec
[ 4] 1.00-2.00 sec 101 MBytes 844 Mbits/sec
[ 4] 2.00-3.00 sec 103 MBytes 865 Mbits/sec
[ 4] 3.00-4.00 sec 105 MBytes 879 Mbits/sec
[ 4] 4.00-5.00 sec 107 MBytes 898 Mbits/sec
[ 4] 5.00-6.00 sec 82.8 MBytes 695 Mbits/sec
[ 4] 6.00-7.00 sec 78.2 MBytes 656 Mbits/sec
[ 4] 7.00-8.00 sec 84.4 MBytes 708 Mbits/sec
[ 4] 8.00-9.00 sec 88.9 MBytes 746 Mbits/sec
[ 4] 9.00-10.00 sec 94.5 MBytes 793 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 932 MBytes 782 Mbits/sec sender
[ 4] 0.00-10.00 sec 931 MBytes 781 Mbits/sec receiver


iMacWifi >> NAS

Code:
Accepted connection from 192.168.1.132, port 54059
[ 5] local 192.168.1.45 port 5201 connected to 192.168.1.132 port 54060
[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 51.8 MBytes 434 Mbits/sec
[ 5] 1.00-2.00 sec 59.5 MBytes 500 Mbits/sec
[ 5] 2.00-3.00 sec 60.4 MBytes 507 Mbits/sec
[ 5] 3.00-4.00 sec 60.9 MBytes 511 Mbits/sec
[ 5] 4.00-5.00 sec 61.3 MBytes 514 Mbits/sec
[ 5] 5.00-6.00 sec 61.9 MBytes 519 Mbits/sec
[ 5] 6.00-7.00 sec 62.2 MBytes 521 Mbits/sec
[ 5] 7.00-8.00 sec 62.8 MBytes 527 Mbits/sec
[ 5] 8.00-9.00 sec 62.1 MBytes 521 Mbits/sec
[ 5] 9.00-10.00 sec 61.5 MBytes 516 Mbits/sec
[ 5] 10.00-10.01 sec 441 KBytes 443 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 605 MBytes 507 Mbits/sec receiver
 

borisko

Neophyte
Joined
Jun 27, 2019
Messages
9
Possibly an important note, I tried running iperf3 from MacOSvm to NAS as UDP to test that as well, and that seems to work:

MacOSvm >> NAS (via UDP)



./Downloads/iperf3 -c 192.168.1.45 -u -b 100m -t 10 -i 1 -f m

Connecting to host 192.168.1.45, port 5201

[ 4] local 192.168.1.229 port 55382 connected to 192.168.1.45 port 5201

[ ID] Interval Transfer Bandwidth Total Datagrams

[ 4] 0.00-1.00 sec 10.8 MBytes 90.4 Mbits/sec 1381

[ 4] 1.00-2.00 sec 12.0 MBytes 100 Mbits/sec 1530

[ 4] 2.00-3.00 sec 11.9 MBytes 99.7 Mbits/sec 1521

[ 4] 3.00-4.00 sec 12.0 MBytes 100 Mbits/sec 1532

[ 4] 4.00-5.00 sec 11.9 MBytes 100 Mbits/sec 1525

[ 4] 5.00-6.00 sec 11.9 MBytes 99.7 Mbits/sec 1519

[ 4] 6.00-7.00 sec 12.0 MBytes 100 Mbits/sec 1534

[ 4] 7.00-8.00 sec 11.9 MBytes 99.9 Mbits/sec 1520

[ 4] 8.00-9.00 sec 12.0 MBytes 99.9 Mbits/sec 1530

[ 4] 9.00-10.00 sec 11.9 MBytes 99.6 Mbits/sec 1521

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams

[ 4] 0.00-10.00 sec 118 MBytes 99.0 Mbits/sec 0.056 ms 1421/15112 (9.4%)

[ 4] Sent 15112 datagrams


Important packet loss by the way, 9.4% ? That's extremely weird... Anyone has an idea ?
 

seanm

Senior Member
Joined
Jun 11, 2018
Messages
468
Probably not too helpful, but I run several macOS VMs with VMWare (hosted on a Mac), and don't have these issues.
 

borisko

Neophyte
Joined
Jun 27, 2019
Messages
9
They connect properly to your NAS box yeah? Thanks for the input anyway. I ordered a very affordable double port Intel NIC, hopefully it will help the problem. If not, I might wipe the whole system and start from scratch, have my eyes on a cheap Power edge T110 II that should be more suitable as well for the nas itself if any of this starts being too hard.
 

borisko

Neophyte
Joined
Jun 27, 2019
Messages
9
I completely changed my system, using proper server hardware this time. The intel NIC i purchased turned out to be a fake (and still a Realtek...) BUT I tried reinstalling on a Dell Poweredge T110 II equiped with a Broadcom Limited NetXtreme BCM5722 gigabit ethernet port, and I still am experiencing exactly the same issue. At this point I don't know what to do.

When trying different config for iperf, in particular adjusting the TCP window value, I do get somewhere. So I believe I am getting closer to identifying the issue, but as I am a total noob in Network configs, I have no idea how to fix it.

Please see test below that seems to work:

Code:
./iperf3 -c 192.168.1.28 -w 80000
Connecting to host 192.168.1.28, port 5201
[  4] local 192.168.1.192 port 50015 connected to 192.168.1.28 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  73.3 MBytes   614 Mbits/sec                 
[  4]   1.00-2.00   sec  74.0 MBytes   621 Mbits/sec                 
[  4]   2.00-3.00   sec  73.2 MBytes   614 Mbits/sec                 
[  4]   3.00-4.00   sec  73.5 MBytes   617 Mbits/sec                 
[  4]   4.00-5.00   sec  73.9 MBytes   620 Mbits/sec                 
[  4]   5.00-6.00   sec  73.6 MBytes   618 Mbits/sec                 
[  4]   6.00-7.00   sec  73.0 MBytes   612 Mbits/sec                 
[  4]   7.00-8.00   sec  72.6 MBytes   609 Mbits/sec                 
[  4]   8.00-9.00   sec  73.3 MBytes   615 Mbits/sec                 
[  4]   9.00-10.00  sec  73.2 MBytes   614 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   734 MBytes   615 Mbits/sec                  sender
[  4]   0.00-10.00  sec   734 MBytes   615 Mbits/sec                  receiver

iperf Done.


But when trying larger value for window size, it fails as usual:

Code:
./iperf3 -c 192.168.1.28 -w 100000
Connecting to host 192.168.1.28, port 5201
[  4] local 192.168.1.192 port 50018 connected to 192.168.1.28 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  86.5 MBytes   726 Mbits/sec                 
[  4]   1.00-2.00   sec  79.7 MBytes   669 Mbits/sec                 
[  4]   2.00-3.01   sec  32.9 MBytes   275 Mbits/sec                 
[  4]   3.01-4.01   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   4.01-5.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec                 
iperf3: error - control socket has closed unexpectedly


Also tried a fix following this thread which seemed similar to my problem: https://www.ixsystems.com/community...ransfer-aborts-on-smb-share-mac-client.69553/
I added a tunable with Set Variable=net.inet.tcp.reass.maxqueuelen, Value=4096, Type=Sysctl. Tried Rebooting, nothing, still same issue. I also tried a quick fix to run the following command with various sizes:
Code:
sysctl net.inet.tcp.reass.maxqueuelen=80000
net.inet.tcp.reass.maxqueuelen: 8192 -> 80000


Still same issue, and here is regular iperf test after all that (without any options set to the command):

Code:
./iperf3 -c 192.168.1.28
Connecting to host 192.168.1.28, port 5201
[  4] local 192.168.1.192 port 50025 connected to 192.168.1.28 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.01   sec  38.5 MBytes   321 Mbits/sec                 
[  4]   1.01-2.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec                 
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  38.5 MBytes  32.3 Mbits/sec                  sender
[  4]   0.00-10.00  sec  38.4 MBytes  32.2 Mbits/sec                  receiver

iperf Done.


I'm really stuck at this point, and badly need help :-(

Every other computer connected on the network via wireless has no issue, I'm the only one with problems thru wired connection...
 

borisko

Neophyte
Joined
Jun 27, 2019
Messages
9
I tried something silly tonight, just to see if that would help: Setting the Rate limit to 80MB/s on the VM hardware network through the Proxmox GUI. Since it looked like a saturation of the network from the beginning before (but at the same time I had perfectly working hypervisor > Freenas iperf results) I thought it could help me identify the origin of the issue or even fix it partially.

It seems like it did ! I have more consistent transfers now, albeit some times it still hangs, it seems a bit more reliable (see iperf results at the bottom). At this point, I'm not sure yet that it isn't a misconfiguration in my hypervisor, the freenas box or the router, or a simple bottleneck.

So new question now... Where is my bottleneck? Any test I should perform to identify further? Is it the router, my single Ethernet Gigabit port shared with the host, or an I/O problem of some sort? I have limited pci-e availability but if adding an Intel double gigabit card that I passthrough just to that VM will fix my crazy problems for 30$, I'd gladly do that. Your opinion is gladly welcomed!

Code:
./iperf3 -c 192.168.1.28
Connecting to host 192.168.1.28, port 5201
[  4] local 192.168.1.192 port 55983 connected to 192.168.1.28 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  54.0 MBytes   453 Mbits/sec                 
[  4]   1.00-2.00   sec  63.6 MBytes   534 Mbits/sec                 
[  4]   2.00-3.00   sec  65.0 MBytes   546 Mbits/sec                 
[  4]   3.00-4.00   sec  63.5 MBytes   532 Mbits/sec                 
[  4]   4.00-5.00   sec  62.0 MBytes   521 Mbits/sec                 
[  4]   5.00-6.00   sec  66.8 MBytes   560 Mbits/sec                 
[  4]   6.00-7.00   sec  64.6 MBytes   543 Mbits/sec                 
[  4]   7.00-8.00   sec  62.5 MBytes   524 Mbits/sec                 
[  4]   8.00-9.00   sec  63.2 MBytes   530 Mbits/sec                 
[  4]   9.00-10.00  sec  66.2 MBytes   555 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec   632 MBytes   530 Mbits/sec                  sender
[  4]   0.00-10.00  sec   631 MBytes   529 Mbits/sec                  receiver
 

borisko

Neophyte
Joined
Jun 27, 2019
Messages
9
I found something that seemed to fix my issues. Note that I changed for the Intel pro/1000 NIC that I received this morning, passed through directly to the VM since my last post, and still was experiencing the same exact problems (actually worse, since I couldn't set the max connection to 80MB/s like before, and it would now kill the network card and needed a reboot to regain any internet access...).

Since overloading or saturation seemed to be the issue, I researched further on that. It appears the issue is from OSX itself (who would have thought... ;-)), and TSO no less (TCP Segmentation Offload) which seems to overload any NIC not properly configured, especially on Hackintoshs and MacOS VMs.

In other words... It was NEITHER a Freenas issue, NOR a proxmox/qemu/KVM issue, but a MacOs one... Potentially related to the age of my inboard card, but probably not as I was also experiencing it when using virtualized one.

Hence, for posterity, steps for the fix:

Verify that your TSO is actually on (on by default in Mojave apparently)

Code:
sudo sysctl -a | grep net.inet.tcp.tso


This should output:

Code:
net.inet.tcp.tso: 1


Run this command in OSX terminal (to set TSO off temporarly):

Code:
sudo sysctl -w net.inet.tcp.tso=0


Check if the problem persists.

If it is fixed, you can add that command to run at startup by either editing your sysctl.conf or creating a new one (works with any text editor, or via command line), adding the following line, and saving it to /private/etc/sysctl.conf :

Code:
net.inet.tcp.tso: 0



I am now thinking of purchasing a second Intel pro/1000 card for the NAS box as well, but my double Realtek NIC configured as a link aggregated interface surprisingly seems to work fine (fingers crossed...).
 
Top