VMware Alternatives

[rvassar]

I had been running ESXi 6.x with a free license in my home lab happily for years, but they started aging out the CPU support faster than I could deal with, and in attempting to keep up, I hit the wall on an upgrade with it getting really picky about a Supermicro BIOS update. When ESX 6.7 EOL'ed I threw in the towel and gave Proxmox a try. I had a clustered solution running between a test node and an old laptop, using TrueNAS Core as shared storage, in less than an hour. I then exported my Bhyve VM's off TN Core and got live migration working. I've since played with various parked quorum votes, so I can turn off the bigger machines, and even have a weekend ZFS replication task running to virtualized TrueNAS Scale VM with passthru disks. (which is very sketchy... they are not actual passthru disks, no SMART, etc... I'm experimenting, its not production...)

I need to remedy my two remaining production jails, after which I can migrate my NAS to Scale. If Scale had the ability to live migrate VM's & containers in the GUI, I would probably consider it. Otherwise, Proxmox does everything I need of it. (caveat: I don't run Windows VM's...) The one thing that may force me to reconsider is inflationary energy pricing. I'm looking at a 20 - 30% bump in $/kWh here in a few weeks when my contract expires. That may have me looking at Proxmox on a couple Raspberry Pi's...

 

[CJRoss]

Since I work with(for?) one of those customers that VMWare expects is too big to leave, I don't think it's a huge surprise that they're staying with VMWare. From what I understand, a large portion of the astronomical price hikes that people have been talking about are due to VMWare simplifying their licensing and including a lot more things in the license.

If you were a small shop only using a few pieces of VMWare, your price will go up by multiples due to all of the additional things that you would now be getting. If you were a large shop already using most of the VMWare product stack, you only saw an increase of 10-20%. I don't deal directly with the purchasing or licensing, but that's what I've gathered from asking around.

Personally, I stopped using VMWare at home when I went full linux. I've had XCPng running for a while and I'm in the process of standing up Proxmox as I find them each useful depending on my use case.

 

[WI_Hedgehog]

Hey TrueNAS Community! With the ever-changing landscape, we wanted to run a survey to find out which is your most likely candidate as an alternative to VMware? Have one that isn’t listed? Let us know below!

A large percentage of member issues are related to VM usage. For stability and reliability, TrueNAS is best run on bare-metal with no VMs under it, those are best served by a VM server.

Thank you for asking the question.

 

[CountBuggula]

I had been running ESXi 6.x with a free license in my home lab happily for years, but they started aging out the CPU support faster than I could deal with, and in attempting to keep up, I hit the wall on an upgrade with it getting really picky about a Supermicro BIOS update. When ESX 6.7 EOL'ed I threw in the towel and gave Proxmox a try. I had a clustered solution running between a test node and an old laptop, using TrueNAS Core as shared storage, in less than an hour. I then exported my Bhyve VM's off TN Core and got live migration working. I've since played with various parked quorum votes, so I can turn off the bigger machines, and even have a weekend ZFS replication task running to virtualized TrueNAS Scale VM with passthru disks. (which is very sketchy... they are not actual passthru disks, no SMART, etc... I'm experimenting, its not production...)

I need to remedy my two remaining production jails, after which I can migrate my NAS to Scale. If Scale had the ability to live migrate VM's & containers in the GUI, I would probably consider it. Otherwise, Proxmox does everything I need of it. (caveat: I don't run Windows VM's...) The one thing that may force me to reconsider is inflationary energy pricing. I'm looking at a 20 - 30% bump in $/kWh here in a few weeks when my contract expires. That may have me looking at Proxmox on a couple Raspberry Pi's...

I've been running a vSphere cluster in my homelab for over a decade using the VMUG license, which for my purposes was worth the $200/year costs. I too, however started running into problems with VMware dropping support for the hardware I was running, particularly Fibre Channel cards. I had to upgrade because VMUG only included the latest and one major version back, but couldn't keep throwing new hardware at the problem.
I've played around with Proxmox some, but Scale seemed like a perfect solution back when they were advertising the clustering features for VMs and containers as "coming soon". Sadly they don't even seem remotely interested in pursuing that sort of functionality anymore and it's clear that they're definitely not working on it. Even if Scale does someday get the ability to live migrate VMs and containers, it won't be in the next few years.
If Proxmox had any sort of integrated kubernetes support I'd have gone that route ages ago. I was about to settle on installing Rancher on top of Proxmox when I discovered Harvester. Hopefully that will work out better than Scale.

 

[awasb]

bhyve (with vm-bhyve)

 

[joeschmuck]

A large percentage of member issues are related to VM usage.

Maybe it would be a better statement "A large number" or "quite a few members running non-server hardware". Proxmox (not that you were inferring any specific VM platform) was not so stable a few years ago, I still don't know how stable it is but I see much less of it these days. ESXi seems to be extremely stable.

TrueNAS is best run on bare-metal with no VMs under it, those are best served by a VM server.

"Can I get an Amen !" Absolutely no argument from me here.

However the real issue is someone running a Type-1 Hypervisor, TrueNAS as a VM, and then running a VM on TrueNAS such as Windoze, something that requires a lot of resources. I do not feel that is wise. I can see many people do not allocate enough resources. But there is one caveat that is okay... running a small VM's like Plex on TrueNAS that is a VM on a Type-1 hypervisor, I don't see as an issue, low resources.

Hopefully I made a little sense, I'm trying to multitask and I fail at it often.

 

[usergiven]

This makes me sad because I've been running free ESXI since 2016 with first FreeNAS and now Core in a VM with zero stability issues following community recommendations. I know it won't go away tomorrow but when I end up getting new hardware I will need at least two hosts, one of them TN on bare metal. The last time I checked even Proxmox didn't quite provide the stability that ESXI gave when passing through an HBA to a VM.

I think it's time to dip my toe into Scale again to see if some of the kinks have been worked out since this time last year.

 

[TN68]

Other: Nutanix

 

[Snafu]

I just performed my migration of ~30 VM’s from VMWare to XCP-ng/XO with using my TrueNAS Scale server as my iSCSI datastore. Went pretty smooth.

 

[danb35]

A large percentage of member issues are related to VM usage.

"Large percentage" is admittedly ambiguous, but if it means "double digits," i.e., >= 10%, I really doubt this is the case. It's certainly a non-zero percentage, but it's certainly far short of a majority.

But be that as it may, I didn't take the question to be asking about the hypervisor software on which people were running TrueNAS, but rather more generally. Though why the hell this poll is in the Announcements forum I have no idea.

 

[xCatalystx]

Since I work with(for?) one of those customers that VMWare expects is too big to leave,

Have you/they considered splitting your workloads across 2 platforms? We fall into that category as well - but are seriously considering running across 2 platforms to reduce cost.

I would say >90% of our workloads can be provisioned completely automatically (inc data migration logic) - so I don't think it would be to difficult (even for a test).

 

[TXAG26]

How well does XCP-NG handle PCIe pass-through for something like a LSI 3008 SAS HBA?

 

[awasb]

HBAs are generally speaking well supported (with VT-x/VT-d or SVM/IOMMU enabled). And they are quite easy to integrate, e.g.

Code:

1. lspci
2. /opt/xensource/libexec/xen-cmdline --set-dom0 "xen-pciback.hide=(0a:00.0)(0a:00.1)"
3. reboot
4. xl pci-assignable-list
5. xe vm-param-set other-config:pci=0/0000:0a:00.0,0/0000:0a:00.1 uuid=VM_UUID

There is only one minor restriction AFAIK (concerning GPUs):

While Intel iGPUs and AMD radeon cards (almost) all work, only some NVidia cards are supported: some Quadro / Tesla / Grid. GeForce is not (officially) supported. To activate accelerators (encoder/decoder ASICs - AMD VCE/nvidia NVENC) you need a real screen attached to the machine or some "dummy" stick. Something like this:

https://www.amazon.de/gp/product/B0747HC25C/ref=as_li_qf_asin_il_tl?ie=UTF8&tag=clptutorials-21&creative=6742&linkCode=as2&creativeASIN=B0747HC25C&linkId=e31619605824d7e5cdd95b746b249221

 

[pwol]

... hi everyone
i will switch over to Oracle-Linux-Virtualization-Manager
supposedly no calmer way to get a supported path to linux-kvm ;-)
btw in veeam user groups the discussion goes this way

 

[FrankWard]

I just performed my migration of ~30 VM’s from VMWare to XCP-ng/XO with using my TrueNAS Scale server as my iSCSI datastore. Went pretty smooth.

Is the migration done via XCP or did you have to use Solarwinds or some other tool to do the conversion?

 

[probain]

Is the migration done via XCP or did you have to use Solarwinds or some other tool to do the conversion?

XCP has quite new built-in tooling to migrate VMs from vmware :)

 

[stbtom]

TrueNas Core of course :)

 

[TXAG26]

Just tried XCP's ova import tool through the Xen Orchestra web GUI interface and it is pretty slick - drag & drop. Worked well for a 5GB ova that I moved over.

 

[CJRoss]

Have you/they considered splitting your workloads across 2 platforms? We fall into that category as well - but are seriously considering running across 2 platforms to reduce cost.

I would say >90% of our workloads can be provisioned completely automatically (inc data migration logic) - so I don't think it would be to difficult (even for a test).

I have zero control or say over the infrastructure. However, I'm not sure what you think they would be gaining by this.

 

[Stux]

FWIW, I migrated away from ESXI some time ago to bhyve with TN-Core. Someday I'll probably migrate to Scale too.