Slow Performance - NFS on Windows and CIFS/SMB on Linux

[angelkiller2]

So I'm having some performance issues with my FreeNas box. I've created a basic two drive mirror. I make two datasets. I share one with NFS, the other with CIFS. Windows can copy to the CIFS share at 100+ MB/s. Windows then copies to the NFS share at ~25 MB/s. Linux (Debian Testing) copies to the NFS share at 100+ MB/s. Then Debian copies to the CIFS share at about 35 MB/s.

The environment is fairly simple, I don't have anything crazy going on. The FreeNAS box is a HP Microserver Gen 8. 12 GB (ECC) RAM, 4x 3TB Seagate Constellation ES.2 drives. The Windows box is running Windows 10 Pro. The Linux machine is a laptop running Debian Testing. All three devices are connected to an unmanaged gigabit switch. The test file is a DVD iso file.

I can't think of anything that would cause this. No wifi, no powerline, no small files, no encryption, no VLANs, etc. The fact that one protocol on each machine operates at normal speeds indicates that it's not a network issue, but more of a protocol issue. I'm not really sure what I should check or change to resolve that.

I wanted to keep this short and to the point. I can go in plenty more detail. Just tell me what you need to know or what commands I should run.

 

[angelkiller2]

I'm going to bump this because I think this is a very simple problem.

• Two hard drives in a mirror
• Two datasets
• One shared with CIFS/SMB, the other shared with NFS.
• Windows machine: 100+ MB/s transfer speed on CIFS, ~20 MB/s transfer over NFS.
• Linux machine: 100+ MB/s on NFS, 35 MB/s on CIFS.

All machines are wired to a unmanaged gigabit switch. The test file is a DVD image.

Since both machines *can* reach 100+ MB/s, the problem seems to lie with the sharing protocol and not the hardware or network.

I'm happy to provide any additional information.

 

[bigphil]

I wonder if the Windows client is issuing sync commands when transferring over NFS. To test you can try setting the zfs sync option to sync=disabled on the dataset in question. You can get the current value by: "zfs get sync yourpool/yourdataset" and then set it by: "zfs set sync=disabled yourpool/yourdataset" just for testing. As for the Linux client, I'm not sure about that one...maybe someone else will have an idea. Also...please always post what version of FreeNAS you're running and all system specs.

 

[angelkiller2]

I wonder if the Windows client is issuing sync commands when transferring over NFS. To test you can try setting the zfs sync option to sync=disabled on the dataset in question. You can get the current value by: "zfs get sync yourpool/yourdataset" and then set it by: "zfs set sync=disabled yourpool/yourdataset" just for testing.

Thanks! I set sync as disabled and now I'm the transfer jumped up to ~70 MB/s. A significant improvement.

Some questions:

1. Can you tell me a little about what the sync command is and why turning it of helps?

2. Why is this not mentioned anywhere? Is my setup not standard?

3. The improvement is significant, but still not full speed. Any other ideas why the transfer is limited to 70 MB/s?

Also...please always post what version of FreeNAS you're running and all system specs.

Sure thing.

I'm running FreeNAS-9.10.2-U1 (86c7ef5).

The FreeNAS system is a HP Microserver (Gen 8). CPU is upgraded to a 3.1 GHz dual core Xeon. 12 GB ECC DDR3 RAM. Hard drives are 3TB Seagate Constellation ES.2.
My Win10 box is a 4.0 GHz Ci5 6600K, Gigabyte GA-Z170N-Gaming 5 motherboard, 16 GB DDR4, GTX 1060, Samsung EVOs for storage.
My Linux machine is a Lenovo X260 (2.5 GHz Ci7, 8 GB DDR4, 256 GB Samsung SSD) running Debian Testing.

 

[SweetAndLow]

what is the nic on the clients? Also are you using cifs-utils on the linux client or smbfs? You should always use cifs-utils and make a mount point on linux for best performance.

 

[angelkiller2]

what is the nic on the clients? Also are you using cifs-utils on the linux client or smbfs? You should always use cifs-utils and make a mount point on linux for best performance.

The FreeNas system has a HPE 332i (which I think is a Broadcom NetXtreme BCM5720). The Win10 machine has a Killer E2400 (which appears to be a variation of a Qualcomm Atheros AR8161 chipset). Linux laptop has an Intel I219-V.

So I installed cifs-utils with apt-get. I tried transferring the DVD image again and I am hitting 120 MB/s according to the file manager. This is great!

Previously (before I installed cifs-utils), I had been still been mounting the share with mount. Smbfs is not installed. Do you know what I was using for CIFS? Or how I could find out?


Now I have one protocol that both Windows and Linux can access at full speed (CIFS). This solves most of my problem. Now, I'd still like to figure out why NFS on Windows maxes out at ~70 MB/s.

 

[SweetAndLow]

Because windows isn't designed to use NFS. Or the driver needs to be updated.

Sent from my Nexus 5X using Tapatalk

 

[bigphil]

Thanks! I set sync as disabled and now I'm the transfer jumped up to ~70 MB/s. A significant improvement.

Some questions:

1. Can you tell me a little about what the sync command is and why turning it of helps?

2. Why is this not mentioned anywhere? Is my setup not standard?

1) If that's all you changed, then it looks like Windows was issuing sync commands for NFS writes. This tells the storage to commit the write or flush cache (depending on what Windows is actually calling) to persistent storage. With sync disabled you're essentially bypassing that and as soon as FreeNAS has the data in memory it issues a call back to the client saying the write was commited (even though it wasn't). That's the gist of it. There is much more detailed info on the net about it if you want to read up on it.

2) Depending on what your use case is, it's generally best to keep sync=standard. The safest approach is to use sync=always so all writes are commited to storage so there is less chance of corruption if something goes wrong (power outage, crash, etc.). This is where having a fast NVMe or SSD drive setup as a SLOG device comes into play so performance can be acceptable again. Again...tons of info on the net about how this setting works if you want to dive into it.

 

[Stux]

And if you have a SLOG, then it should have Power Loss Protection. That's if you care about your data, and if you didn't you could just disable sync writes.

 

[moosport]

I wonder if the Windows client is issuing sync commands when transferring over NFS. To test you can try setting the zfs sync option to sync=disabled on the dataset in question. You can get the current value by: "zfs get sync yourpool/yourdataset" and then set it by: "zfs set sync=disabled yourpool/yourdataset" just for testing. As for the Linux client, I'm not sure about that one...maybe someone else will have an idea. Also...please always post what version of FreeNAS you're running and all system specs.

I found this thread while trying to fix my slow linux client to SMB shares issue. Windows client to SMB transfers at over 100MB/s, But linux client transfer rate is half at 50 plus MB/s. Performed a few tests for baseline. Plugin linux client using the Windows client network cable and transfer is still slow. Linux client to NFS and transfer rate is over 100MB/s. So its not a cable issue.

Found a thread on ubuntu forum explaining how the issue can be resolved. If you are using Nautilus and not mounting the share, you will encounter this problem. By mounting the share manually or through fstab resolved the problem.

https://ubuntuforums.org/showthread.php?t=288534