Slow writes to TrueNAS 12.0-u7. 1G to 10G connection

sabur

Cadet
Joined
Aug 21, 2019
Messages
5
I have a Hypervisor server with Linux VMs. The VM is capped at 1G speeds as it's using a "VM" interface on the hypervisor and the NAS itself has multiple SMB shares all on a 10G fiber.

The NAS specs:
2630v3L
96GB ram
HBA H310 Mini IT Mode
8x 7200 RPM 6TB disks
ZFS Raid2
10G fiber to 10G port on switch for SMB Shares

I am mounting the cifs share via fstab and I'm using this command to write a test file.
Code:
dd if=/dev/zero of=/mnt/test/test1.img bs=1G count=1 oflag=dsync


With that I get about 37mbps writes.

Is this not supposed to be faster? Closer to 100mbps? There's not really a whole lot of data being written from other VM's to the datastore so I don't think there's anything causing performance issues. The dataset that's being written to has sync and everything else disabled. It does have LZ4 compression

For the Network:
pfsense
VLAN for VM's and another VLAN for the nas
Theres one switch for the whole setup.
The switch is layer 2

What I want to know is there something that could be potentially causing the slow writes? I understand with the setup for the NAS that it's not the fastest in the world, but shouldn't it be "faster"? Let me know if there's anything else I can post here for clues or questions about anything else on my setup.
 

sabur

Cadet
Joined
Aug 21, 2019
Messages
5
Also forgot to mention that the NAS's 10G fiber MTU settings are at 1500 should I change that to 9000?
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
You're only getting 37 megabit per second writes? Why do you think you would only get 100 megabit per second writes on a system capable of 1 gigabit?

Perhaps you meant MBytes/sec?

Terminology and units matter. Please review the following


and make sure you're using the correct units in order to help us understand your problem better. Also, please describe your hypervisor setup, as not all hypervisors are created equal.

The dataset that's being written to has sync and everything else disabled.

It has sync ... enabled? Disabled? You could read this sentence either way. If sync is enabled, then I'd say there's a good chance this is working as designed.

Typically, async writes to a RAIDZ2 pool with lots of free space would be limited to somewhere between the speed of one of the underlying devices, and maybe half the aggregate speed of the component devices. There are a lot of factors in play. I would expect the gigabit interface to become a choke point.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Top