CIFS transfer @ 50-60Mbps

[jgreco]

iperf results being lower than transfer speeds would be a red flag, likely signalling pebcak ....

 

[cyberjock]

And FYI, SMB 3.0 is now out(not yet supported in FreeNAS though), so the rules for the road with regard to CIFS are changing yet again. Still want to use a network transfer protocol as a means of benchmark comparison? ;)

 

[Knowltey]

Ah, okay, seems we're actually on the same page. I was just saying it wasn't and end all be all of possible speed, but still a good indicator for troubleshooting. I use it all the time and have been for a year or two now since I discovered it. I was more or less just noting that it is possible to have more network throughput than iPerf.

 

[Knowltey]

Sorry for the double post, for some reason it won't let me edit my post. But heh, ran iPerf again and get 850, so whatever was going on earlier must've just been a temporary issue anyhow.

 

[cyberjock]

yeah, if you have more network throughput than iperf there's many possible reasons including:

1. you invalidated the test because of another limitation in the network hardware.
2. you don't understand the test sufficiently to interpret the results and determine what you did wrong.
3. The test give bogus incorrect numbers and since you know it isn't #1 or #2 it must be an indicator of a network issue.
pseudo #4 - iperf doesn't work

I can tell you that 99.99% of people here don't have the knowledge and experience to prove #1(sad, but true), 99.99% of people will never accept #2 because they always think they know everything or that it can't be so complex they don't already understand it, and everyone assumes it must be #3 or #4 because of their lack of knowledge(no offense to anyone in this thread).

I will freely admit that when it comes to iperf I stick to the standard values because I don't have a good grasp of the deep level fundamentals. The standard values seem to work for everyone in relation to troubleshooting FreeNAS performance issues(which is all I care about in this forum). All that crap with window size, buffer length, and the other parameters if improperly used for your network can cause tests that make you think you'll never see 1MB/sec+ ever again. So I just press the "i believe" button and use the defaults as it works pretty much all of the time. It is also why we allow for more "fudge factor" and consider 850Mb/sec as a good test. If we were actual networking nazis we'd setup the proper parameters with iperf and then realize something is wrong if we got below 970Mb/sec.

Doing benchmarks is a freakin' art. I don't care what anyone says or how much of a bada** you think you are. People don't think it is because they see so many websites that do benchmarks(many of them have very bad testing parameters, why do you think so many conflict with each other?). And I hate the fact that so many people think its so simple. It's not. It's very complex. I just BBQed a user 2 weeks ago in this forum because he posted a long benchmark that had pretty graphs and stuff. It looked very well thought out, very well planned and very well executed. But if you looked closer he had a pool of 6 disks in a RAIDZ2 and some of the benchmarks claimed 2GB/sec+, 500k IOPS/sec+, and he didn't have the necessary skills/knowledge to realize he isn't skilled/trained enough to even be posting benchmarks for the world. Anyone with 1/2 a brain should immediately realize that 2GB/sec is physically impossible, that 500kIOPS/sec+ is impossible. Sadly, since that guy made his post I've had more than 5 people message me asking how to get those kinds of numbers because they get nothing like that and they want to know what the did wrong. To that guy I can only say "thanks alot a**hole". LOL.

Now I need to go kill some hookers in GTA now that I'm pissed off again.

 

[Knowltey]

Lol, I figured it was #2 for me since all I really know about iperf is starting it with iperf -s on on machine and then running iperf -c to the other machine.

 

[MtK]

I rebooted both client & server and re-did the test with a direct cable between them (no switch/router in the middle).
iperf still tops at 50...

but I did noticed this:

Code:

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
4828 root 3 103 0 30532K 3252K RUN 1 5:13 200.00% iperf

maybe it affects the results?


OK, so I can maybe answer myself - NO.
I just tested the client with 2 instances of IPERF one as client an one as server, and it still tops at 50... :(

and this is the freenas-to-freenas:

Code:

# iperf -c 192.168.1.10 -P 1 -i 1 -p 5001 -f k -t 10
------------------------------------------------------------
Client connecting to 192.168.1.10, TCP port 5001
TCP window size: 42.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.10 port 10478 connected with 192.168.1.10 port 5001
[ ID] Interval      Transfer    Bandwidth
[  3]  0.0- 1.0 sec  1686016 KBytes  13811843 Kbits/sec
[  3]  1.0- 2.0 sec  29824 KBytes  244318 Kbits/sec
[  3]  2.0- 3.0 sec  31360 KBytes  256901 Kbits/sec
[  3]  3.0- 4.0 sec  33280 KBytes  272630 Kbits/sec
[  3]  4.0- 5.0 sec  29696 KBytes  243270 Kbits/sec
[  3]  5.0- 6.0 sec  33024 KBytes  270533 Kbits/sec
[  3]  6.0- 7.0 sec  30720 KBytes  251658 Kbits/sec
[  3]  7.0- 8.0 sec  33408 KBytes  273678 Kbits/sec
[  3]  8.0- 9.0 sec  31232 KBytes  255853 Kbits/sec
[  3]  9.0-10.0 sec  26368 KBytes  216007 Kbits/sec
[  3]  0.0-10.0 sec  1965056 KBytes  1605913 Kbits/sec

 

[Knowltey]

Oh didn't notice this before, but you're also the same person having an issue with your hard drive tempoeratures? I guess I'll share a troubleshooting experience I had a while back that may or may not help your situation. Anyway one day I went in and dusted out the inside of my FreeNAS, but after putting it back together and booting it up my CIFS transfer speed were like half what they usually were, which was odd since I didn't change anything about the build. Noticed the fan I have blowing into my hard drive bay wasn't running, forgot to plug it back in. Did that and rebooted and transfer speeds were back to normal. The temperatures without the fan still weren't overheating by any means (34C) but still apparently enough to reduce their performance by a decent amount.

(Although if your iPerf it topping out at 50 there's probably a different issue at play, just figured I'd give you that tidbit to try)

 

[MtK]

well, I have no idea how did this happen (while iperf still tops at 50), but transfering a file to my PC (the same client as before) starts at 118MB/s and drops to 90-95.

from my PC to FN it's even faster and stable @ 110MB/s... :D

 

[eraser]

iperf has been an industry standard for testing throughput for quite a few years. I have yet to see a situation where iperf was wrong in its assessment on the forums.

The default settings of iperf sometimes need to be changed to get the best throughput.

See below... I got a 16% improvement in measured Bandwidth just by changing the TCP Window Size command line parameter on both my iperf server and iperf client:

Code:

[root@bfd] ~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  4] local 192.168.120.14 port 5001 connected with 192.168.120.50 port 59781
[ ID] Interval      Transfer    Bandwidth
[  4]  0.0-10.0 sec  940 MBytes  788 Mbits/sec
 
[root@bfd] ~# iperf -s -w 256k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  256 KByte
------------------------------------------------------------
[  4] local 192.168.120.14 port 5001 connected with 192.168.120.50 port 59797
[ ID] Interval      Transfer    Bandwidth
[  4]  0.0-10.0 sec  1.09 GBytes  939 Mbits/sec

MtK should be able to show even better iPerf performance improvements as his posted test was done with a tiny 8 KB TCP Window Size...