FreeNAS on ESXi - Thoughts and ramblings

[Scharbag]

To preface, this is for my personal home server only, not for business etc. Total overkill but hey, I like overkill.

So, I have recently gone through the task of migrating from FreeNAS and VirtualBox to ESXi and FreeNAS. It was an entertaining exercise and I certainly learnt a lot. Here are some of the key points to consider if you choose to do this:

1. VT-d is required for hardware passthrough
2. ensure that you have lots of RAM :)
3. SSD based ZIL/SLOG really really really improves the performance of a spinning media based pool (thanks tips:))
4. I prefer to share storage to ESXi via NFS over ISCSI
   • this is due to the fact that I found ESXi datastore browser and file tools slow and unreliable as many times when I moved VMs it would fail or take forever - never had errors on FreeNAS side, such as iSCSI timeouts etc.
   • NFS allowed me to move VMs via the FreeNAS shell at a much faster rate - rsync is so wonderful
   • with SSD ZIL, the write performance is not terrible over NFS but be careful, cheap SSD drives are poor performers compared to enterprise SSDs (Intel DC S3XXX family seem to be good performers)

Of real note though, I think that I may go back to FreeNAS with VirtualBox. My reasons being:

1. I still use plugins under FreeNAS because they just work, file sharing is so slick and I am lazyish
2. I can run the same types and counts of VMs using VirtualBox
   • primarily Windows 7 32bit VMs
3. No longer need to use NFS or iSCSI to share data to hypervisor
   • I assume that FreeNAS shares files to jails with some native BSD magic that is faster than NFS/iSCSI
4. All of the CPU resources are available to FreeNAS when needed
   • again, overkill :)
5. I can devote all of my ram to FreeNAS and have VirtualBox use only what it needs when it needs it instead of reserving RAM for FreeNAS (in my case, there is ~30GB unused RAM available but it may be needed so I cannot reserve it to FreeNAS)
6. ESXi free version cannot use Veeam (what a silly silly silly restriction)
   • I use snapshots and replication under ZFS to backup VMs, so all is good, but I wanted to learn Veeam
7. Free version of ESXi does not have any of the really cool features that make it stand apart from VirtualBox, such as storage vMotion, HA etc.

So, I will contemplate my next moves here and see which way I want to go. I am also excited that FreeNAS 10 will have Bhyve.

Now, I just need to see how hard it will be to swap back to using VirtualBox with the VMs that I have running now. I can always use OVF/OVA but that is time consuming. Wonder if FreeNAS would be unhappy with me running VirtualBox jail when FreeNAS is virtualized in ESXi... Or would that be like dividing by zero...

Anyhoodles, the above is my ESXI/FreeNAS/VirtualBox ramble. Please let me know if I am missing something.

Cheers,

 

[joeschmuck]

I am very happy with my ESXi setup. I'm now running two ESXi systems, one specifically for my firewall (because the hardware isn't compatible with Sophos UTM), and the main ESXi machine is for FreeNAS and several other VMs.

I'm using XSi Backup to automate my VM backups and it works very well. It's simplistic. I don't care for the way it needs to be configured, it could have some tool which makes it a bit easier to manage but it works.

Your ramblings has me curious about VirtualBox, I've never really used it.

In my defense, I'm not a power user these days, my main interests for my home network is to keep all the computers at home virus free and everything just working. I like to tinker but on my schedule, not when someone is telling that the internet is broken again. I do like running VMs in VMWare Workstation but since I don't feel like spending a ton of money to upgrade it, free ESXi seems to fit the bill. I did loose some features going this route but I've managed.

Maybe I need to give VirtualBox a shot.

Thanks for rambling, it did me some good.

 

[pirateghost]

Virtual box on FreeNAS is a hack. You shouldn't rely on it. I certainly wouldn't dream of running my VMs on it. Never have and never will think that Virtual box is any more than a desktop VM solution, for spinning up a quick VM or 2. It can never replace a true hypervisor.

You're really comparing 2 different types of hyper visors too. You should give Proxmox a shot.

 

[Scharbag]

Yeah, I hear ya on the types of hypervisors. I had been running VBox on FreeNAS for a quite some time without any issues. As said, this is for home use and I only run simple VMs that would only affect my ability to watch TV if they failed... It was also one of the reasons that I wanted to try ESXi.

I am really looking forward to Bhyve on FreeNAS 10. Proper hypervisor on top of FreeNAS. Just wondering how much work it will be to change to FreeNAS 10 when it comes out. Lots of things to learn, that will be for certain.

Cheers,

 

[m0nkey_]

Bhyve is already available 9.10, known more as iohyve.