vmxnet3.ko for FreeNAS 9.X in ESXi 5.5

Status
Not open for further replies.

David E

Contributor
Joined
Nov 1, 2013
Messages
119
The vmxnet3.ko in VMware's ESXi 5.5 tools does not work with FreeNAS 9.2.X or 9.3, I have recompiled the (unmodified) source code for this driver, and am making available a ready to go kernel module for the FreeNAS versions listed below. You will want to temporarily use the E1000 NIC type so you can copy the driver into FreeNAS, then change it to VMXNET3 on the next boot.

For instructions on how to install the driver, see the latter part of the following post:
http://itanalyses.blogspot.com/2013/02/jumbo-frames-in-virtual-freenas-831-on.html

Instructions for compiling yourself
Rough guide of the necessary steps, substitute 9.2.1-RELEASE below with the FreeNAS version you are interested in. Updated to include ajohnson's faster build below:
  • Set up a FreeBSD 9.2 VM in VirtualBox
  • Create and attach a second hard drive to the VM (20G or so)
    • Find the disk, ls /dev look for ada*
    • zpool create tank ada1
  • Install Dependencies
    • cd /usr/ports/devel/git
    • make install clean
    • cd /usr/ports/sysutils/cdrtools
    • make install clean
    • cd /usr/ports/ports-mgmt/pbi-manager
    • make install clean
  • Check out FreeNAS
    • cd /tank
    • git clone git://github.com/trueos/trueos.git
    • cd trueos
  • Find version to build against
    • git tag -l
    • git checkout 9.2.1-RELEASE
  • Unpack vmware tools
    • scp the tools to the vm in /root (vmware-freebsd-tools.tar.gz)
    • cd /root
    • gunzip vmware-freebsd-tools.tar.gz
    • tar xf vmware-freebsd-tools.tar
    • tar xf /root/vmware-tools-distrib/lib/modules/source/vmxnet3.tar
  • Build
    • cd /tank/trueos
    • make buildenv
    • cd /root/vmxnet3-only
    • SYSDIR=/tank/trueos/sys make
Resulting kernel module will be in /root/vmxnet3.ko

Determining your kernel version for FreeNAS 9.3
As of 9.3 FreeNAS has moved to a much more frequent release process, that is revving the kernel across patches, but not updating the overall version number. To determine what version of the TrueOS kernel you are using, you'll need to find the MANIFEST file for the version of FreeNAS you are using. Start here: http://download.freenas.org/9.3/ pick the release train, ie STABLE, then the date of the ISO you installed, then the MANIFEST file. Inside it look for the line:

git@gitserver.ixsystems.com:/git/repos/freenas-build/trueos.git 4663c83

The last group of letters and numbers indicates the git revision, and corresponds to a revision at the Github repository https://github.com/trueos/trueos

Armed with this revision, follow the same instructions above for building, but instead of checking out a tag revision, check out the exact revision for your build, ie:

git checkout 4663c83

Available Kernel Modules
  • ESXi 5.5 FreeNAS 9.3-STABLE-201503270027 (Git rev e006356) shasum: 0ec8c6b4992432cc438a1f03b201eac10645eabe
  • ESXi 5.5 FreeNAS 9.3 x64 shasum: 4ea6e8dc8069fbcbb735599bf76286bb8a58b721
  • ESXi 5.5 FreeNAS 9.2.1.6 x64 shasum: ff156321029d71f353f8752878a2e6d157f93f22
  • ESXi 5.5 FreeNAS 9.2.1.3 x64 - use the module from 9.2.1.1
  • ESXi 5.5 FreeNAS 9.2.1.1 x64 shasum: 378cf7b6d4267991b31e412cc90ddc4b1a6afb59
  • ESXi 5.5 FreeNAS 9.2.1 x64 shasum: 69baa7d0621ee604277cb6768a504f79cb72b872
  • ESXi 5.5 FreeNAS 9.2.0 x64 shasum: 831f99c967713c88a02fa240c2542cf3b41cbd7d
 

Attachments

  • vmxnet3-ESXi-5.5-FreeNAS-9.2.0-x64.zip
    26 KB · Views: 885
  • vmxnet3-ESXi-5.5-FreeNAS-9.2.1-x64.zip
    25.2 KB · Views: 868
  • vmxnet3-ESXi-5.5-FreeNAS-9.2.1.1-x64.zip
    25.9 KB · Views: 991
  • vmxnet3-ESXi-5.5-FreeNAS-9.2.1.6-x64.zip
    15.3 KB · Views: 902
  • vmxnet3-ESXi-5.5-FreeNAS-9.3-x64.zip
    15.3 KB · Views: 1,171
  • vmxnet3-ESXi-5.5-FreeNAS-9.3-STABLE-201503270027.zip
    15.3 KB · Views: 819
Last edited:

ser_rhaegar

Patron
Joined
Feb 2, 2014
Messages
358
Quick and easy to load, thanks!
 

FREAKJAM

Dabbler
Joined
Oct 10, 2013
Messages
20
Can someone verify that it also works with FreeNAS 9.2.1 & ESXi 5.5?
I'm getting "no configured interfaces were found" after booting :(
 

AkumaKuruma

Cadet
Joined
Feb 12, 2014
Messages
1
this will only work with FreeNAS 9.2.0. a fresh compile needs to be made against 9.2.1 to make it work there.

I would definitely be interested in a write up of the steps you are taking to create the file. some of us would like to be able to compile other variations or need to do an in house compile for QA reasons.
 

David E

Contributor
Joined
Nov 1, 2013
Messages
119
Ya I'm working on that, as well as a compile for 9.2.1, however Github is currently checking out a multi-100 MB repository to me at ~30KiB/s.. so it'll be awhile :|
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
I know several people would appreciate a how-to on compiling this stuff. This is often asked for in various fashions. Usually it's someone using "less than the latest version" of FreeNAS with the "less than the latest version" of ESXi. In my case, I'm using 9.2.0 with ESXi 5.1. ;)
 

David E

Contributor
Joined
Nov 1, 2013
Messages
119
Added a module for 9.2.1 x64, and rough instructions for doing it yourself. Its possible I missed steps since I made the instruction list a couple weeks after doing it myself, let me know and I'll be happy to update the original post.

@cyberjock: You may want to try the 5.5 tools build with 5.1, they tend to try to stay pretty backwards compatible with the OS-level drivers.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
And the 5.5 vmxnet3.ko does work with 5.1. Woohoo. Not too useful for my purposes. But it does work!
 

FREAKJAM

Dabbler
Joined
Oct 10, 2013
Messages
20
I can confirm the latest vmxnet3.ko works with FreeNAS 9.2.1/ESXi 5.5. Thanks alot!
 

ajohnson

Dabbler
Joined
Feb 25, 2013
Messages
18
Thanks for documenting all of this David. I have a compile setup on my end for this as well. It's working quite nicely.

I do have one question -- when a FreeNAS update occurs, do you do a full make clean / make release cycle or have you worked out some way of git pulling and compiling updates without needing to do a make clean (would be much faster in theory).
 

David E

Contributor
Joined
Nov 1, 2013
Messages
119
I was planning on testing this tonight for the 9.2.1.1 release, I think I can just stash/pull --rebase/stash pop on the parent repo, delete the os-(I forget the name here) build directory, and fetch/checkout the new release tag for the two sub repos. I'll give it a go and report back.
 

ajohnson

Dabbler
Joined
Feb 25, 2013
Messages
18
Great, thanks much.

Also, I wanted to contribute the method I was using before I read this thread. It actually used 'make buildenv" to set the environment up so that it builds using the FreeNAS/FreeBSD toolchain in such a way that you can build the driver outside of the source directory. I don't know that this is really more useful than your method, but I'll put it here in case it is of interest. I guess the one benefit of this approach is that you don't actually need to build the full FreeBSD kernel / FreeNAS release if you don't want to -- you can just build the kernel module.

(Before performing these steps, follow David's instructions up to and including "Check out FreeNAS". I use a slightly different directory for the FreeNAS source below, so just consider that.)

Instructions for creating a vmxnet3 driver for FreeNAS from outside the FreeNAS build directory:
Code:
# Set the environment up to use the FreeNAS/FreeBSD build toolchain instead of the system's default toolchain:
cd /usr/local/freenas/FreeBSD/src
make buildenv
 
# Assuming your vmware-tools-distrib is extracted in a directory structure as below, do the following:
cd /usr/local/vmware-tools-distrib/lib/modules/source
tar xvf vmxnet3.tar
cd vmxnet3-only
SYSDIR=/usr/local/freenas/FreeBSD/src/sys make
# The .ko file should now live in /usr/local/vmware-tools-distrib/lib/modules/source/vmxnet3.ko

 

David E

Contributor
Joined
Nov 1, 2013
Messages
119
So fetching then checkout of the new tag, combined with deleting the os-base directory worked fine for me, and I added 9.2.1.1.

I also confirmed building your way worked great, I figured there was probably a far cleaner way to build just what was needed, but my FreeBSD knowledge is pretty limited so I went with the hacky method :)
 

ser_rhaegar

Patron
Joined
Feb 2, 2014
Messages
358
Are you guys cloning the FreeBSD source into the FreeNAS directory? I was following David's instructions however there is no FreeBSD folder under FreeNAS nor any submodule in the FreeNAS git that I can see.
 

spidernas

Dabbler
Joined
Mar 26, 2014
Messages
10
I've been trying over the past few days to compile the module myself as well for 9.2.1.3, but without success. Seems that some paths have changed as well as the build environment.

If anyone could help with updated instructions or a new binary module posted, I would really appreciate it.

Thanks.
 

David E

Contributor
Joined
Nov 1, 2013
Messages
119
I recompiled for 9.2.1.3 and it had the same shasum as 9.2.1.1, so I assume there were no kernel changes that affected its compilation, so feel free to use 9.2.1.1 for 9.2.1.3.

wrt to compilation, probably the easiest thing to do at this point is to directly git clone the kernel source at https://github.com/trueos/trueos, check out the tag for the release, then follow ajohnson's build instructions.
 

ddimick

Contributor
Joined
Feb 23, 2013
Messages
144
Annoyingly the 9.2.1.3 update appears to have blown away the module in /boot/modules. Keep a copy of it somewhere else on the NAS locally or you'll wind up adding an E1000 again just to get it back on there.
 

ser_rhaegar

Patron
Joined
Feb 2, 2014
Messages
358
Annoyingly the 9.2.1.3 update appears to have blown away the module in /boot/modules. Keep a copy of it somewhere else on the NAS locally or you'll wind up adding an E1000 again just to get it back on there.
Every update will blow away any file system changes to the root file system.
 

pbucher

Contributor
Joined
Oct 15, 2012
Messages
180
Every update will blow away any file system changes to the root file system.

Very easy workaround...I've been doing this since version 8.3. Download the ISO and attach it to the VM and upgrade using the ISO image. It keeps the files in /boot/modules every time.

Also the stock binaries with 5.5 work great with 9.2.1.2 and newer. I've pumped several TB of data through them using jumbo frames.
 
Status
Not open for further replies.
Top