FreeNAS 8.3, jumbo frames and vSphere 5.0

[madel35]

I have successfully installed FreeNAS 8.3.0 release in a vSphere 5.0 host.

A physical machine ( Windows 2008 R2 ) is running Windows Server Backup, using an iSCSI target, kindly offered by FreeNAS.

As shown in network graph, performance is quite good ( more than 60 MiB/s on average ), but I was asking myself if it can do better using jumbo frames.



The problem is that I can't get frames bigger than 1500 between the peers, even configuring mtu=9000 on every station between the two:

• mtu=9000 in FreeNAS interface em0
• mtu=9000 in vSphere vSwitch connected to FreeNAS virtual machine
• mtu=9000 in D-LINK DGS-3120 L2 switch
• mtu=9000 in Windows 2008 R2 physical NIC

After doing some RTFM, I read about virtual NIC types on VMWare KB; so I gave a try to VMXNET 3 virtual NIC.

Unfortunately VMXNET 3 isn't detected by FreeNAS...

Did you get jumbo frames with FreeNAS inside vSphere hosts? If yes, how?

 

[cyberjock]

Honestly, I wouldn't go to jumbo packets unless you are absolutely sure that all of the devices on your network support it. There's a discussion in another thread from the last 24 hours discussing this. If you are 100% sure that ALL of your network devices(switches, printers, computers, etc) then great and you should keep going. But if there is any question you shouldn't do it. I had minimal performance gains with FreeNAS from my experience with jumbo packets. Windows Server Backups does do some compression, so that is likely your limiting factor.

Also, having an Intel NIC for the FreeNAS server helps. You didn't mention exactly what your NIC is so I'll just put that out there.

 

[madel35]

Well, I'll check out the ongoing discussion.

I'm 100% sure that all devices support jumbo frames, with the exception being the E1000 driver used by FreeNAS virtual machine.

According to VMWare docs, a guest os can get jumbo frames if virtual machine version is a recent one ( mine is version 8 ), VMWare Tools are installed ( FreeNAS sports "3rd party" VMWare tools ), and virtual NIC is one among E1000, VMXNET 2 Enhanced and VMXNET 3 ( available from the drop-down in the "add hardware" wizard ).

It seems that only VMXNET 2 Enhanced and VMXNET 3 paravirtualized NICs can do jumbo frames, but FreeNAS doesn't detect them ( I tried VMXNET 3 ).

Windows Server Backup does some compression, and CPU could be a limiting factor: I'll try with two cores @2.2 GHz, instead of one only.

Thank you for support :)

 

[pbucher]

You need to install the VMXNET3 driver. Check out the post at http://forums.freenas.org/showthread.php?9316-VMXNET3-support&highlight=vmxnet3 for more details. Also remember to change the MTU on your vSphere switch.

That all said I've had problems when I changed the mtu to 9000. I've been using a vSAN with FreeNAS and the vmxnet3 driver since the early 8.3 betas with great success. Its been a while since I tried changing the mtu size again, I've wanting to get some good solid up time and make sure things are stable before I try more tweaks.

 

[madel35]

Thank you, that was really what I was looking for :)

I'll make it work for sure.

--Marco

 

[madel35]

It worked perfectly :D

Here it is the interface graph for the shiny new interface vmx3f0:



There are two runs of Windows Server Backup: first a full backup, and then an incremental one.

Higher transfer rate means higher cpu usage; FreeNAS is acting as an iSCSI target: