PotatoInside
Cadet
- Joined
- Nov 29, 2018
- Messages
- 2
Im using FreeNAS as VM storage and it is connected directly to the VM host with a 10Gb Ethernet link. When doing sequential writes over NFS I only get 10-15MB/s while reads are around 120MB/s. If I copy the same file with SFTP, writes are 120MB/s too so it clearly isnt a disk speed issue.
I have tested everything I can think of and all threads with the same issue are related to ESXi which I am not using.
IPerf shows 9Gb/s through 10Gbe and 985Mbps through 1Gbe, I have connected to the NAS through both nics and NFSd and SMBd cpu usage is well under 1% on both machines. sftp and nfs to nvme only volume run about 400r/400w MBps which isnt great but is a problem for another time.
All speed benchmarking has been done using dd if=localfile of=test.img bs=4M status=progress
The hardware is as follows:
NAS:
- I7-2700K
- Mellanox Connect-X2 10Gbe
- 3x segate st4000dm000 in raidz1
- samsung 960 evo nvme (separate pool from platter drives)
- 16GB ddr3 1600
VM Host:
- 2x Xeon x5670
- 72GB ram
- Connect-X2 10Gbe
The network setup is pretty simple, 10Gbe is directly connected between the two servers with mtu=9000 and a 1Gb link is connected to the rest of the network.
I can provide any extra info if it is needed.
Edit: I forgot to mention this is on FreeNAS 11.2-RC2
I have tested everything I can think of and all threads with the same issue are related to ESXi which I am not using.
IPerf shows 9Gb/s through 10Gbe and 985Mbps through 1Gbe, I have connected to the NAS through both nics and NFSd and SMBd cpu usage is well under 1% on both machines. sftp and nfs to nvme only volume run about 400r/400w MBps which isnt great but is a problem for another time.
All speed benchmarking has been done using dd if=localfile of=test.img bs=4M status=progress
The hardware is as follows:
NAS:
- I7-2700K
- Mellanox Connect-X2 10Gbe
- 3x segate st4000dm000 in raidz1
- samsung 960 evo nvme (separate pool from platter drives)
- 16GB ddr3 1600
VM Host:
- 2x Xeon x5670
- 72GB ram
- Connect-X2 10Gbe
The network setup is pretty simple, 10Gbe is directly connected between the two servers with mtu=9000 and a 1Gb link is connected to the rest of the network.
I can provide any extra info if it is needed.
Edit: I forgot to mention this is on FreeNAS 11.2-RC2