SOLVED 500kbs transfer speed?

[Leary]

Today I uptraded my NAS.

It had 8 GB ram and 2xwd red.
Today I changed the ram to 16 gb, added a PCI card providing 4 new sata ports and added 3 new drives (2x wd blue and 1xhgst, 4tb each). I deleted the old shares, datasets and volumes, wiped all the disks and chose raidz2 for the new volume, added a dataset and a windows share.

Before adding the disks, I copied files with about 60mb/s from my PC to the nas. Now, its only 500kb /s, so not even 1 mb per second!

I was really shocked and tried out different things:
-Copying with windows
-Copying with TeraCopy
-Copying with FreeFileSync
-Restarting the NAS
-Turning compression from lz4 to off
-Trying to copy from my laptop instead my PC
-Copying via Wifi / Lan cable

nothing changed the copying speed.

System:
Build FreeNAS-9.10.1 (d989edd)
Platform Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz
Memory 16050MB

Any idea what I could have done wrong?

 

[joeschmuck]

The first thing I'd do is connect your FreeNAS LAN port directly to your main computer with a new/different Ethernet cable and try the transfers again. Rule out the Ethernet interface. Do not use WiFi, that will be terribly slow.

Also when copying a file, just move a single file which is at least 200MB and report the speed. Lastly, how full is your pool? (screen shots work nicely)

There are also other tests to ensure the internal FreeNAS speed is correct but I'm fairly certain that you have an Ethernet issue.

EDIT: On your Ethernet ports on the motherboard, ensure the wire pins are not crossed. They should all be separated.

 

[Leary]

I have tried connecting the nas directing to the PC, but I cant access it with that. Is there something special I have to do?
After connecting, I started it and instead of the DHCP lan IP it normally takes, it chose 0.0.0.0
After typing that into my explorer or browser, it didnt find anything. Same with the old IP.

Also, I haven't changed anything concerning the cables when adding the drives and before adding them, the speed was much higher

Which are the other tests you mentioned?

Edit: Here is a screen of my pool: http://www.bilder-upload.eu/show.php?file=0ca4bb-1471633026.png

 

[joeschmuck]

You can do it a few different ways, I like Static IPs but for this case lets keep it simple. During this process do not reboot or shutdown your FreeNAS server.

1) Connect your system like normal and power everything up.
2) Log in to the FreeNAS GUI and note the IP address.
3) Move your Ethernet cables around to achieve a direct connection.
4) Verify the GUI is responsive.
5) Try to copy some files.

If this doesn't work then you will need to assign static IP addresses to FreeNAS and likely your main computer.

 

[Leary]

This is actually very weird and I apologise for the trouble.
Now it is copying with 80mb/s without having changed anything.

Probably the reason was the following:
I am just transferring all my backups to the nas since I had to wipe all nas drives today for creating a bigger volume.
I have this one folder containing mostly ebooks, which is 60gb big and contains about 30.000 files. There are some 2gb files in there but most files must be less than a mb. It seems like it was copying about 6 hours with a speed of 500kb/s and now where this bit is over and the files are larger, it got back to 80mb/s. Still odd that it dropped so dramaticly but now it seems back to normal.
Thanks you very much for your help!

 

[joeschmuck]

It's called drive latency. When a file is written the head must move to write the actual data and then is must record on the drive where it put that data. Writing lots of small files, even if they were all 1K each, means the heads move around all over the place and this takes a lot of time. Writing large files takes less time (well really transfers data faster) because the head moves much less frequently.

This is why I asked you to copy a single large file at least 200MB in size to see what the transfer speed was.

 

[Leary]

yes, that must have been the problem.
do you think its a problem that I turned off compression during the copying process?
Would you recommend to turn it on again now where I copied some files with compression enabled and some with compression disabled?

 

[joeschmuck]

Compression has nothing to do with it, it's drive latency. Google it. Compression is something different.

 

[Leary]

Yes, what I mean is not related to the copying speed:
At the beginning I used compression for some GB. Then I turned compression of for some other GB to see if it affects something.
So half of the date in the dataset is compressed an the other one not. I just wanted to make sure that is not a problem for longterm use somehow to have half of the data compressed and the other half not

 

[Nick2253]

If anything, the compressed data would write faster, because it has to write less data to the disk. The time to compress data, especially with a modern processor, is insignificant next to the drive latency.

 

[Arwen]

I'd also recommend sorting your eBooks directory / folder so that it does not have 30,000
files in it. One simple way is to use alphabetic, then author and then book(s) associated with
that author. This is roughly what I have;

v

Jules Verne

Mysterious Island
20,000 Leagues Under the Sea
...​

It will take time, but future additions can be properly sorted as they are added.

ZFS does better than some file systems when having large number of directory / folder entries.
But, a CIFS / SMB share can still have problems with such a large number of entries.

 

[anodos]

I'd also recommend sorting your eBooks directory / folder so that it does not have 30,000
files in it. One simple way is to use alphabetic, then author and then book(s) associated with
that author. This is roughly what I have;

v

Jules Verne

Mysterious Island
20,000 Leagues Under the Sea
...​

It will take time, but future additions can be properly sorted as they are added.

ZFS does better than some file systems when having large number of directory / folder entries.
But, a CIFS / SMB share can still have problems with such a large number of entries.

Rapid-fire thoughts on this issue:

• You can get better samba performance on such a dataset by creating the dataset with case-sensitivity turned off. Then add the samba share auxiliary parameter "case sensitive = true". This basically turns off samba's case-sensitivity compatibility layer, which can dramatically improve performance on directories containing tons of files.
• Upping the amount of RAM in your system will also improve performance when writing lots of small files. ZFS uses a portion of your RAM as a write cache (transaction groups). This scales up as the amount of RAM in your system increases.
• Having a good network card can make a difference with your write speed in stressy situations.
• Samba write speed can become CPU-bound when doing lots of small writes (you will see a single thread at 100% during the write).