Comparing with OmniOS, lower write performance.

[x-cimo]

I have been using OmniOS with the napp-it web gui for a few years now.

Since then FreeBSD/FreeNAS has made significant progress in the ZFS world and use SMB3 (OmniOS is stuck in the legacy SMB world).

I run my ZFS appliance in a VM running on a ESXi 5.5 Update 1 Hypervisor. I passthrough my JBOD cards to the VM, give it 20GB of RAM (the system has 64GB), I also use vmxnet3 10G NIC.

I've installed FreeNAS (9.2.1.5 x64) without issue, i've copied the vmxnet3 drivers. Imported my pool, everything went well (amazing!). NFS and SMB works too.

So feature wide it seem to be good to go.

The only issue I encountered is SMB write speed.

Basically I imported a pool from my OmniOS, enabled SMBv3. Then started to write a 20GB file from a SSD on my workstation.

Number look as follow:

20GB from ZFS to SSD:
OmniOS: 103Mb/s
FreeNAS: 100Mb/s

20Gb from SSD to ZFS
OmniOS: 114Mb/s
FreeNAS: 80Mb/s

Is anyone aware of any setting or tweak or things I could look at to figure out why there is such a difference? (nearly 40 Mb/s) on the same pool. Same configuration, two different OS.

Thanks!

 

[x-cimo]

I should have mentioned the zpool configuration I'm using: it's 2x raidz1 of 3* 4TB driver. (total of 6 4TB drives)
I also remember that I did one tweak to the OmniOS vm which is to:
on OmniOS: edit /etc/system and add:
set zfs:zfs_vdev_max_pending=2


However I don't know where I can set this in FreeNAS?

 

[Ericloewe]

FreeNAS is quite picky about ZFS pools. Ideally, you should use a pool created in FreeNAS. It could be having a hard time writing to the pool.

 

[x-cimo]

That would be unfortunate. I will see if I can find another solution as there is too much data on the pool to move it out.

 

[bestboy]

This thread suggests there is an issue with the current SMB3 implementation. Maybe you can test with SMB2 instead.

 

[deed02392]

SMB is implemented in the kernel on OmniOS vs FreeNAS' userland implementation. This will result in a negative performance difference on FreeNAS.

 

[Spearfoot]

Thanks for your post! Many of us are intrigued by OmniOS and napp-it; it's interesting to hear from users who have experience with them.

I run my ZFS appliance in a VM running on a ESXi 5.5 Update 1 Hypervisor. I passthrough my JBOD cards to the VM, give it 20GB of RAM (the system has 64GB), I also use vmxnet3 10G NIC.

How are you passing the disks through? You'll probably get much better results passing through an HBA rather than individual disks as VMDKs.

I've installed FreeNAS (9.2.1.5 x64) without issue, i've copied the vmxnet3 drivers. Imported my pool, everything went well (amazing!). NFS and SMB works too.

Which version of FreeNAS are you using? The latest version in the 9.x train is 9.10.2-U5

You say that you copied the vmxnet3 drivers? That seems odd -- FreeNAS 9.10.x comes with support for vmx built-in, so you shouldn't have to copy a driver.

 

[Ericloewe]

Hey, @Spearfoot, the thread is ancient...

 

[Spearfoot]

Hey, @Spearfoot, the thread is ancient...

Doooohhhhh!