Awful NFS/SMB write performance

Status
Not open for further replies.

PotatoInside

Cadet
Joined
Nov 29, 2018
Messages
2
Im using FreeNAS as VM storage and it is connected directly to the VM host with a 10Gb Ethernet link. When doing sequential writes over NFS I only get 10-15MB/s while reads are around 120MB/s. If I copy the same file with SFTP, writes are 120MB/s too so it clearly isnt a disk speed issue.
I have tested everything I can think of and all threads with the same issue are related to ESXi which I am not using.
IPerf shows 9Gb/s through 10Gbe and 985Mbps through 1Gbe, I have connected to the NAS through both nics and NFSd and SMBd cpu usage is well under 1% on both machines. sftp and nfs to nvme only volume run about 400r/400w MBps which isnt great but is a problem for another time.
All speed benchmarking has been done using dd if=localfile of=test.img bs=4M status=progress

The hardware is as follows:
NAS:
- I7-2700K
- Mellanox Connect-X2 10Gbe
- 3x segate st4000dm000 in raidz1
- samsung 960 evo nvme (separate pool from platter drives)
- 16GB ddr3 1600
VM Host:
- 2x Xeon x5670
- 72GB ram
- Connect-X2 10Gbe

The network setup is pretty simple, 10Gbe is directly connected between the two servers with mtu=9000 and a 1Gb link is connected to the rest of the network.
I can provide any extra info if it is needed.

Edit: I forgot to mention this is on FreeNAS 11.2-RC2
 
Joined
Jul 3, 2015
Messages
926
My initial thought is that NFS being 'sync write' is that its syncing writes and you don't have a dedicated SLOG for this so it's using part of your disk pool which would be slow.

You could probably do with a small fast SSD SLOG device. Search for the forums for info and recommendations on this.
 

PotatoInside

Cadet
Joined
Nov 29, 2018
Messages
2
Thanks, its a step in the right direction. With a 12GB SLOG I am now getting 50-60MB/s, still not the same performance as SFTP but getting there.
 
Status
Not open for further replies.
Top