To add more to this. I put in enhancements to windows driver performance and the 10gbe network tunables for freenas.
Connecting iperf on my FreeNAS 9.10 SuperMicro x8dtn+ box, 2 quad 2.83ghz xeons, to a win10 i7-hexcore processor with 32G memory, running iperf 2.0.5. Both machines running Intex x540-t2 10GBE cards
I did manage to max out my 10GBe line, but it took 15 concurrent streams. With 8 concurrent streams, it hit 1097 with each stream averaging about 135 mb/s
15 streams-maxed out line:
Code:
D:\iperf-2.0>iperf -c 192.168.1.49 -p 5001 -f M -t 10 -P 15
------------------------------------------------------------
Client connecting to 192.168.1.49, TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
[ 10] local 192.168.1.180 port 52668 connected with 192.168.1.49 port 5001
[ 17] local 192.168.1.180 port 52675 connected with 192.168.1.49 port 5001
[ 16] local 192.168.1.180 port 52674 connected with 192.168.1.49 port 5001
[ 15] local 192.168.1.180 port 52673 connected with 192.168.1.49 port 5001
[ 13] local 192.168.1.180 port 52671 connected with 192.168.1.49 port 5001
[ 12] local 192.168.1.180 port 52670 connected with 192.168.1.49 port 5001
[ 14] local 192.168.1.180 port 52672 connected with 192.168.1.49 port 5001
[ 11] local 192.168.1.180 port 52669 connected with 192.168.1.49 port 5001
[ 8] local 192.168.1.180 port 52666 connected with 192.168.1.49 port 5001
[ 7] local 192.168.1.180 port 52665 connected with 192.168.1.49 port 5001
[ 5] local 192.168.1.180 port 52663 connected with 192.168.1.49 port 5001
[ 6] local 192.168.1.180 port 52664 connected with 192.168.1.49 port 5001
[ 4] local 192.168.1.180 port 52662 connected with 192.168.1.49 port 5001
[ 3] local 192.168.1.180 port 52661 connected with 192.168.1.49 port 5001
[ 9] local 192.168.1.180 port 52667 connected with 192.168.1.49 port 5001
[ ID] Interval Transfer Bandwidth
[ 17] 0.0-10.0 sec 632 MBytes 63.3 MBytes/sec
[ 16] 0.0-10.0 sec 639 MBytes 63.9 MBytes/sec
[ 15] 0.0-10.0 sec 639 MBytes 63.9 MBytes/sec
[ 13] 0.0-10.0 sec 881 MBytes 88.1 MBytes/sec
[ 12] 0.0-10.0 sec 892 MBytes 89.2 MBytes/sec
[ 14] 0.0-10.0 sec 648 MBytes 64.8 MBytes/sec
[ 11] 0.0-10.0 sec 851 MBytes 85.2 MBytes/sec
[ 8] 0.0-10.0 sec 650 MBytes 65.0 MBytes/sec
[ 7] 0.0-10.0 sec 644 MBytes 64.4 MBytes/sec
[ 5] 0.0-10.0 sec 860 MBytes 86.1 MBytes/sec
[ 6] 0.0-10.0 sec 634 MBytes 63.4 MBytes/sec
[ 4] 0.0-10.0 sec 861 MBytes 86.1 MBytes/sec
[ 3] 0.0-10.0 sec 966 MBytes 96.7 MBytes/sec
[ 9] 0.0-10.0 sec 649 MBytes 64.9 MBytes/sec
[ 10] 0.0-10.0 sec 842 MBytes 84.1 MBytes/sec
[SUM] 0.0-10.0 sec 11288 MBytes 1127 MBytes/sec
8 streams, almost maxed:
Code:
D:\iperf-2.0>iperf -c 192.168.1.49 -p 5001 -f M -t 10 -P 8
------------------------------------------------------------
Client connecting to 192.168.1.49, TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
[ 10] local 192.168.1.180 port 52631 connected with 192.168.1.49 port 5001
[ 9] local 192.168.1.180 port 52630 connected with 192.168.1.49 port 5001
[ 5] local 192.168.1.180 port 52626 connected with 192.168.1.49 port 5001
[ 8] local 192.168.1.180 port 52629 connected with 192.168.1.49 port 5001
[ 7] local 192.168.1.180 port 52628 connected with 192.168.1.49 port 5001
[ 6] local 192.168.1.180 port 52627 connected with 192.168.1.49 port 5001
[ 3] local 192.168.1.180 port 52624 connected with 192.168.1.49 port 5001
[ 4] local 192.168.1.180 port 52625 connected with 192.168.1.49 port 5001
[ ID] Interval Transfer Bandwidth
[ 10] 0.0-10.0 sec 1360 MBytes 136 MBytes/sec
[ 9] 0.0-10.0 sec 1392 MBytes 139 MBytes/sec
[ 5] 0.0-10.0 sec 1360 MBytes 136 MBytes/sec
[ 8] 0.0-10.0 sec 1378 MBytes 138 MBytes/sec
[ 7] 0.0-10.0 sec 1332 MBytes 133 MBytes/sec
[ 6] 0.0-10.0 sec 1360 MBytes 136 MBytes/sec
[ 3] 0.0-10.0 sec 1420 MBytes 142 MBytes/sec
[ 4] 0.0-10.0 sec 1366 MBytes 137 MBytes/sec
[SUM] 0.0-10.0 sec 10969 MBytes 1097 MBytes/sec
Single stream, I got 321 up to NAS and 354 down
Code:
D:\iperf-2.0>iperf -c 192.168.1.49 -p 5001 -f M -t 10 --dualtest -P 1
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.1.49, TCP port 5001
TCP window size: 0.06 MByte (default)
------------------------------------------------------------
[ 4] local 192.168.1.180 port 55259 connected with 192.168.1.49 port 5001
[ 5] local 192.168.1.180 port 5001 connected with 192.168.1.49 port 34730
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 3208 MBytes 321 MBytes/sec
[ 5] 0.0-10.0 sec 3548 MBytes 354 MBytes/sec
So, it looks thread limited. And confirms my previous finding that single copy speed is not amazing but ok, but if you add more simultaneous copies, they all run well, up to line max speed.
If there is anything else someone wants me to test, let me know