Slow CIFS and FTP speeds

[Michael B]

Hello everyone,

I just built a FreeNAS box, but I'm not getting the performance I expected. CIFS read transfers are at around 25 - 27 MB/s and FTP at 12 MB/s for a 1 GB file.

Here's my setup:

Server:
ASROCK B75M-ITX motherboard
8 GB RAM
Intel i3 3220 (3.3 Ghz) processor
Seagate 2TB 7200 RPM Sata3 HD x 2 (one connected to Sata3 port, one connected to Sata2 port)
ZFS Mirror on two drives, encryption enabled
Build: FreeNAS-8.3.1-RELEASE-p2-x64
Onboard Gigabit Lan: Realtek 8111E

Router:
Asus RT-N16, Tomato firmware
Using Gigabit ports


PC1:
Windows XP Pro
Pentium D 2.8 Ghz, dual-core
Broadcom Netlink Gigabit Ethernet
Sata HD

PC2:
Windows 7
Intel i5
7200 RPM Sata HD

I've been searching the forums trying to find what I could tweak to get CIFS performance up. I tried enabling AIO with minimum read/write sizes of 4096, but that made no difference, so I turned it back off. Send files with sendfile(2) is enabled. I also understand large readwrite is enabled by default.

I've also iperf and I think my figures are low:
bin/iperf.exe -c 192.168.47.143 -P 1 -i 1 -p 5001 -f k -t 10
------------------------------------------------------------
Client connecting to 192.168.47.143, TCP port 5001
TCP window size: 8.00 KByte (default)
------------------------------------------------------------
[1912] local 192.168.47.119 port 2009 connected with 192.168.47.143 port 5001
[ ID] Interval Transfer Bandwidth
[1912] 0.0- 1.0 sec 33856 KBytes 277348 Kbits/sec
[1912] 1.0- 2.0 sec 33880 KBytes 277545 Kbits/sec
[1912] 2.0- 3.0 sec 33848 KBytes 277283 Kbits/sec
[1912] 3.0- 4.0 sec 29912 KBytes 245039 Kbits/sec
[1912] 4.0- 5.0 sec 26776 KBytes 219349 Kbits/sec
[1912] 5.0- 6.0 sec 26584 KBytes 217776 Kbits/sec
[1912] 6.0- 7.0 sec 26624 KBytes 218104 Kbits/sec
[1912] 7.0- 8.0 sec 33880 KBytes 277545 Kbits/sec
[1912] 8.0- 9.0 sec 32760 KBytes 268370 Kbits/sec
[1912] 9.0-10.0 sec 33256 KBytes 272433 Kbits/sec
[1912] 0.0-130.0 sec 311384 KBytes 19617 Kbits/sec
read on server close failed: Connection reset by peer
Done.
I was wondering if the router might be a problem. I tried connecting two PC's directly together with no switch in between and the iperf figures were roughly the same. I'm using a couple of Cat5e cables for short segments and a Cat6 cable for a 11 meter segment. The PC's all report a gigabit connection.

What should I look at to troubleshoot the problem? Thanks!

BTW, I also enabled autotune on the FreeNAS menu and it generated these sysctls:
net.inet.tcp.sendbuf_max 2097152
kern.ipc.maxsockbuf 2097152
net.inet.tcp.recvbuf_max 2097152

 

[Michael B]

Here's an update with a bit more info:

I happen to have Cygwin installed on the Windows PC #2. So I tried to eliminate the Windows PC's hard drive from the equation to see if that improved things. It did. Copying the same 1 GB file from the FreeNAS server to the cygwin PC's /dev/null got about 78MB/s. Much improved.

So perhaps the bottleneck may be with the Windows PCs. I also measured the write speed on PC #2 hard drive with dd:

Code:

dd if=/dev/random of=./tempfile bs=4096 count=262144

and got around 14MB/s. Not sure if 4096 is a large enough bs to test with though.
I measured read speed with

Code:

dd if=./tempfile of=/dev/null

and got around 150MB/s if I recall correctly.

So write performance is significantly slower. This Windows PC has a 4 GB SSD cache built into the hard drive, which might throw off the read results.

I also have Truecrypt encrypting both PC's drives, so I guess that could slow down the performance some. During the transfer, the CPU utilization is only around 25% though on PC #2. It is around 65-75% on PC #1, but not maxed out.

Update:
I tried another dd write test with if=/dev/zero and 16 GB of data to a tempfile and averaged around 35MB/s.

 

[Michael B]

It's fixed now -- mostly. I think some of the ethernet cables weren't plugged in good or something.

Now PC #2 reads at ~85MB/s over CIFS and uploads pretty fast too, I think around ~65 MB/s.
However, PC #1 is still a bit on the slow side, but decent. I'll update with more details later.

Update:
PC #2 is fine. PC #1 reads at about 30 MB/s, but I bought it in 2005, so perhaps I can't expect too much from it.

There's one odd thing about it. I run iperf tests with 64K window and get different results depending on whether it is in server or client mode.

On server mode, iperf running on the PC reports 685Mbits/s. In client mode, it reports 350Mbits/s.