NFS slow, anyone else experience this?

[Johhhn]

Quick info:
Two FreeNAS servers on local gigabit.

With AFP, I can read/write 100MB+ to my laptop from both servers.

On Server B, I NFS Mounted Server A. Doing an rsync I only get 42MB/sec when copying data to Server B.

Anyone notice this? Not too terribly concerned, I'm going to retire my 'test' Server A since I built a 'proper'ish server (B).

thanks guys!

 

[eraser]

I see that you are using rsync. Perhaps this thread is relevant?: http://forums.freenas.org/index.php?threads/rync-to-new-volume-very-very-slow.19230/

 

[Johhhn]

Thanks!!!

What's interesting is that if I do another rsync at the same time (different set of files) both only do around 20MB/sec which suggests the server A hardware is the limitation but I know it's not.

Thoughts?

 

[cyberjock]

Thanks!!!

What's interesting is that if I do another rsync at the same time (different set of files) both only do around 20MB/sec which suggests the server A hardware is the limitation but I know it's not.

Thoughts?

Are you so sure? rsync is single threaded... so yes, it can and does suck if you have multi-TBs of data and are wanting to keep them in sync. The fact that there is no easy tool that can scale up to the quantity of data ZFS uses is precisely why ZFS has its own internal snapshot/replication system. That system has relatively few limits(just like ZFS).

I'm not really sure why you are mounting an NFS share of another server on your server and then doing rsync. That's the back-asswards way of doing it, and defeats one of the main reasons rsync exists... to minimize traffic across the LAN.

The better way to do rsync is to setup rsync just like the FreeNAS manual mentions. Hint: It makes no mention of NFS. ;)

 

[Johhhn]

good point, i was hoping to do a 'quick' copy of files by using rsync instead of cp (wanted to get a progress bar).

i did another test with ftp and it was basically normal fast speeds, so i definitely will chalk that up to rsync via NFS being slow. :D

 

[cyberjock]

FTP can be a dangerous way to move TBs of data.. the FTP protocol has no protection from corrupt packets. I've seen people do backups and realize later that some files were damaged... so FTP isn't the most recommended way to move data...

 

[Johhhn]

Thanks… data wasn't critical, so wasn't too worried :)

Now, I'm on an inexpensive ECC setup :-D running the latest freeNAS.. let's see how that AFP/Time Machine works now ;)

 

[Joshua Parker Ruehlig]

I have been testing with this as well. I can get around 100MBps Read & 44 MB/s write. jumbo frames, rsize/wsize, noatime, made little difference; UDP made things worse. After setting sync=disabled on the dataset I consistently get 100MBps read/write. This is with a 20GB SLC SLOG drive. I'll probably just set sync=disabled for datasets with NFS shares holding non-critical data.

 

[cyberjock]

If you are setting sync=disabled you might as well remove the SLOG.. sync=disabled short circuits the SLOG entirely. ;)

 

[Joshua Parker Ruehlig]

If you are setting sync=disabled you might as well remove the SLOG.. sync=disabled short circuits the SLOG entirely. ;)

I know, it's weird to me I'm so limited even with a slog. I wonder how much slower it'd be without the slog and with sync enabled.

 

[cyberjock]

I know, it's weird to me I'm so limited even with a slog. I wonder how much slower it'd be without the slog and with sync enabled.

That is a million dollar question. ;)

The answer ranges from "you'd never notice" to "you'd want to jump off the nearest bridge". Since an SLOG is virtually only used for sync writes, if you have zero sync writes it won't help. But if it's 100% sync writes, and small ones at that, then the difference can be significant.