[SOLVED] Slow CIFS write speed, fast read speed

[garesu]

Hi all,
I'm just starting with FreeNAS, but I'm trying to read as much as possible, and I'm pretty happy with what it allows me to do compared to my low end QNAP.

The only issue I'm having so far is that while I'm able to saturate the 1 gbps link reading files from FreeNAS, I'm capped at 20MB/s writing to it. I noticed it while transferring data from my old nas to this machine, and started investigating and testing using single >1GB files from my PC's SSD.
The hardware should be plenty powerful to reach that write speed, but just to be sure I started a large file and checked:
top -> CPU never goes north of 5-10%
iostat -x 1 -> %b alternates between 0 and about 50%, averages at about 20%

Network shouldn't be a problem, since I'm able to saturate the link when reading.

I read many threads, which seem to blame CPU, pool performance, not enough ram, network, SMB configuration.
I've also found threads where, especially Cyberjock, advised about not changing parameters, which are now either defaults or unsupported with smb4.

I tried setting CIFS log level to full and checking the log with tail -f /var/log/samba4/log.smbd, which seems to say my windows 10 pc connection is using smb2, while issuing the Get-SmbConnection from powershell yelds

Code:

ServerName ShareName UserName     Credential    Dialect NumOpens
---------- --------- --------     ----------    ------- --------
OLDQNAP    Backups   PC\Giorgio   OLDQNAP\admin 1.5     1      
FREENAS    Public    PC\Giorgio   PC\Giorgio    3.1.1   1       

Any help would be much appreciated
Thanks

 

[Rand]

Good read speed does not necessarily imply good write speed, so I'd check iperf nevertheless.

What kind of files are you writing? Large or small? Edit: a >1GB file.
Are you writing directly from the QNAP? How?, are you using a PC/Mac as intermediate storage?
Deduplication, compression enabled?

In short - some additional detail would be most helpful.

 

[garesu]

The initial copy was done from qnap to freenas, every other test I've made was copying directly to/from the ssd on my PC, so the QNAP is not a factor. I was not able to use iperf, I'd let it run for a couple of minutes without output.
On freenas I ran iperf without parameters, on PC I ran iperf -c freenasip
When I stopped it, it reported speeds in the range of kbps.
Dedup disabled, compression lz4
If any other info is needed, I'll gladly provide, just don't know what would be useful.
Thanks

 

[Rand]

iperf: See this thread post for an example for the correct command https://forums.freenas.org/index.ph...perf-is-showing-1gb-speeds.41218/#post-262786

i.e. iperf -s is needed

 

[garesu]

I was using the wrong iperf version
iperf -s on freenas:
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 204 MBytes 171 Mbits/sec

iperf -s on pc:
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 658 MBytes 552 Mbits/sec

BTW I just reinstalled freenas, so this is after the inital wizard, will try again after I've configured some stuff again

 

[Rand]

Whats the nic in the desktop system btw?

 

[garesu]

It's an Apple thunderbolt to Ethernet adapter, with a broadcom chipset. The PC is an Intel nuc DC3217BY, so it doesn't have LAN and has USB 2 ports only, so thunderbolt is the only way to get gigabit speed

 

[Rand]

Hm ok, and I assume write speeds to the Qnap where in acceptable range?

 

[garesu]

Well, nope, but that's a low end TS-212 with a SoC Marvell Armada dual core 1.2GHz and it's one of many reasons I've decided to replace with something beefier.
Anyway, it is able to go over 30MB/s, better than what I'm achieving with the FreeNAS machine now

 

[Rand]

Ok.
You tried with cifs - have you tried deactivating hostname lookup ? had a case where that caused an issue a couple of weeks back (and another one where it didn't help;))
Have you tried AFP/ftp/ssh,scp?

Just to establish its a protocol issue and not something else?

 

[garesu]

Disable hostname lookups: no improvement
Transfer the same 1.14GB file over FTP with filezilla, same 20MB/s writing speed.

 

[Rand]

So it does not seem to be protocol dependent ...
Do you have any other system that you could use to evaluate the write speed from? Laptop ?

 

[garesu]

Cool fact: FTP reading speed is lower than samba, 60-70MB/s.
An Asus eeepc with USB 2 and a usb3-to-ethernet adapter, so it won't be able to reach gigabit speed but should be able to do 40MB/s. I'll give it a try. Could trying with some Linux live CD on my pc be a good test?

 

[Rand]

If it supports the Thunderbolt Nic, sure

 

[garesu]

Laptop with USB 2 port reads and writes 40MB/s. Will try a distro now, with both thunderbolt and USB adapters

 

[garesu]

I tried different things:
- USB 3 adapter on my PC's USB 2, about 26MB/s (an i3 slower than an Atom, go figure...)
- Ubuntu live CD, usb adapter performance dropped significantly, due to the live CD being on an USB stick
- Ubuntu live CD, thunderbolt adapter, 43MB/s
- Back to Windows again, played with drivers, installed a "non-compatible" driver for a similar adapter. Boom. 68MB/s
Not the 100MB/s I was hoping for, but a good result nonetheless.
Is there something I could do to further improve speed?

 

[Rand]

Get a new PC? :p

There might be some tweaks you can run on the driver end, you can try jumbo frames maybe ... or find yet another driver.
Thunderbolt is fast enough, but maybe the driver support is not up to speed.

Maybe there is another brand adapter with higher rates/better driver support...

 

[garesu]

:D I will, probably in a couple of months. ATM I just have to find a way to stop Windows from falling back to the default driver on every reboot
Thank you very much for your help.

BTW am I supposed to add [SOLVED] to the title now?

 

[Rand]

It will be appreciated - but I don't think most ppl do it;)