ESXi Virtualisation confirmation (BEFORE use)

[Marc Allard]

Hello,

I have read the article in the blog that says that it is possible to run Freenas in ESXi as a guest if I use the vt-d (to pass my LSI port).
http://www.freenas.org/whats-new/2015/05/yes-you-can-virtualize-freenas.html

I have made a test and it is working with my motherboard. I would like to be sure that it is working at 100% before doing it for "real"). Is someone already doing it? should I expect any "problem"?

My motherboard is a Supermicro X10SL7-F


Thank you
Marc

 

[Darren Myers]

according to the manual the Supermicro X10SL7-F does support VT-d but youll need a supported CPU as well. But the real question is, is why do you want to run FreeNAS as a VM?

 

[Marc Allard]

Hello,

My CPU (Xeon E3 1245v3) can use vt-d.
At the moment I use Freenas to handle my disk (Raid Z2 4+2 HDD (3 To each).
I have a few jails but I use some services that run on Windows.
(for example I use crashplan...)
I would like to add the feature dedup of windows (on a Windows 10) so I need a virtual machine (I have a few services running on windows that I use too). For example I would like to use Emby with a GPU to help transcoding (so I need a support of vt-d to pass a graphic card on a virtual machine).
The virtual machine is not very "fast" with virtual box jail (Virtualbox 5 is supposed to be better but I am not good enough to try to put it in Freenas).
As freenas 10 will not come before a long time (bhyve with vt-d), I had the idea to use ESXi, then create a virtual machine with freenas (24-25 Go RAM 4 virtual CPU (2 should probably be enough)) and a virtual machine with Windows (5-6 Go RAM) so I could use vt-d.
But before planning to do it I would like to know if it is really "not dangerous".

Thank you
Marc

 

[gpsguy]

If done correctly, it can be done. But, I wouldn't go so far as saying it's "not dangerous".

You might want to read this thread: https://forums.freenas.org/index.ph...ide-to-not-completely-losing-your-data.12714/ as well as the additional links given in the thread.

 

[Marc Allard]

Hello,

The thread you mention is very interesting I have read it before thinking about virtualisation.
I plan to do a test for performances first (and see if I still keep a correct speed). (I will try in the next week end).
I will continue to have backups (rsync)...
Here is a guide I have seen.
https://b3n.org/freenas-9-3-on-vmware-esxi-6-0-guide/
Thank you for the time
Marc

 

[gpsguy]

I'm glad to hear that you've already read some of the resources on our forums.

I only skimmed through the 2nd URL you posted.

I don't agree with item #13, disabling swap on each disk. The primary reason for FreeNAS allocating swap space on each disk, is to facilitate disk replacements, should the size of the replacement disk, be slightly different that the original disk.

 

[Marc Allard]

Hello,

I was afraid too of the swap part and I didn't plan using it.

Marc

 

[Marc Allard]

Hello,

I have made some tests and here are my findings at the moment.
1) The first part works perfectly. (install...)
2) The NFS is way too slow (480 minutes to import a virtual machine with 25 GB HDD) but ISCSI has a correct speed (less than 10 minutes for the same virtual machine (I wasn't before the computer)).
My speed goes from around 100-107 Mo/sec to a speed of about 90-117 (I have no idea about why sometimes it is faster and sometimes not). At the moment I don't use the new ethernet adaptater (vmxnet3) as I am not sure it is running correctly with freenas (I have installed the guest OS). I am still doing other tests.

Marc

 

[Xelas]

May be WAY too late, but I've been running with this exact setup for almost 2 years now (since 8.something?). NFS is dog-dead-slow because it uses sync writes - this can be partially alleviated with a GOOD SSD SLOG device (one that has battery/capacitor-backed write cache). iSCSI on ESXi does not enforce sync writes, so the performance can be good, but at risk to some data loss. Since I'm not running essential services in my VMs, it's a risk I'm willing to take, but it's still something I plan on fixing once I can scrounge up some dough for an Intel 3500.

I've had between 1-4 VMs in ESXi that utilize iSCSI extents as data stores passed back out to ESXi. There was a time back in the early FreeNAS 9.0-9.1 days when iSCSI was flaky (up to when a single Windows VM would occasionally crash), but it's been solid since about 9.2, and I'm running 3 VMs now with no hiccups.

I've been using the vmxnet3 ethernet driver since forever with no issues that I've noticed. I easily saturate a 1GB link with sequential reads/writes, but without an SSD SLOG, random access and working with large numbers of small files causes occasional slowdowns.

The guide Marc links to above seems excellent - I wish there was something like it 2 years ago! - but I concur that disabling swap seems dumb. I also find his approach of creating a virtual NIC for iSCSI traffic redundant - I've never noticed that to be a bottleneck, but it's something I'm willing to try. It might make breaking the all-in-one box out into separate ESXi / NAS boxes easier later, when I'll get more money, which is never

 

[Marc Allard]

Hello,

Thank you for your post.
I have been using the vmxnet 3 fine but after an update of Freenas I had problems with the network (I wasn't able to keep my network up all the times and I had to reboot the Freenas VM, sometimes after 3 hours, sometimes after one day.)
After that I have installed the vmware driver again and I have added a vmxnet3 only for the iscsi interface (and I have em0 for the freenas share) and I have no problem since then.
I have seen that when I copy (in the VM) from a ISCSI drive to another drive (or directory) it can be really slow (around 20 MB/sec.).
In VMWare web site they talk about one virtual nic too (to increase the speed when there is a problem).
There was also a problem for the speed with the UI of Freenas.
I have added the option listed here
https://bugs.freenas.org/issues/6562
Step by Step
1) Turn off your FreeNAS virtual machine
2) Edit settings on the FreeNAS virtual machine
3) Click on Options Tab
4) Click on General under Advanced
5) Click the Configuration Parameters Button on the bottom right
6) Change hpet0.present from true to false.
7) Click Start
After that the UI of Freenas is a lot faster (as on a true hardware) before it was really slow (10-15 sec.) to make the complete web page appear.
The only problem I have now is for the speed of ISCSI to ISCSI (copy directory to directory). For the other things, it works really great.


Marc

 

[Xelas]

Perhaps it might be worth trying to pass through one of the Intel NICs too, via VT-d, to FreeNAS for shares, and use the other one as the uplink to the virtual switch for the VMs in ESXi? This way, FreeNAS gets its own NIC that isn't virtualized. Hmmm - I now want to try this myself. Of course, this will only work if you use a separate virtual network for ESXi data stores, or you will get a network loop.
You don't strictly need a separate NIC for managing ESXi if you can use a VLAN.

Also, I've never had issues with vmxnet3, but I'm using the one I got from these forums, not the one provided by VMWare, mainly because back in the days of 9.1 the one from VMWare didn't work for some reason that I've forgotten now.
I also have never had the issue of the web interface slowing down
curious to what may cause that, if we're running essentially identical HW and SW.

 

[Marc Allard]

Hello,

I have applied the last update from ESxi6 and now the vmxnet3 is working perfectly (there was a problem with the network that could be lost...)

Marc

 

[jgreco]

Hello,

I have applied the last update from ESxi6 and now the vmxnet3 is working perfectly (there was a problem with the network that could be lost...)

Marc

I enjoy all these threads about all the gyrations people have to go to in order to get vmxnet3 to work. In almost all cases, the E1000 works without trouble and often with similar performance.

 

[cyberjock]

I enjoy all these threads about all the gyrations people have to go to in order to get vmxnet3 to work. In almost all cases, the E1000 works without trouble and often with similar performance.

+1. I enjoy it too.

 

[jgreco]

+1. I enjoy it too.

Don't you have some work to do?