How do 'you' use ESXi?

Status
Not open for further replies.

diedrichg

Wizard
Joined
Dec 4, 2012
Messages
1,319
I'm curious as to how/why a home user would utilize ESXi. How do _you_ use ESXi and what benefits do you find useful?
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
FWIW, I'm a software developer, so I'm not a 'typical home user'...

I use ESXi for virtualization because it's mature, robust, reliable... and free for home use!

In my experience, FreeBSD jails are too lightweight for some of my use-cases, while bhyve is relatively new and unproven. With jails, you are restricted to FreeBSD software packages, which often lag behind the Linux versions; Plex is a good example.

So I use FreeNAS strictly as a rock-solid file server; at which it excels.

Ever notice how many threads here on the forum are posted by users having problem with jails or bhyve? There are quite a few! I seldom have problems with virtualization, because ESXi mostly 'Just Works'. :D

Some of the VMs I run that a home user might also use:
For development work I run these VMs:
  • Oracle database server (11g, 12c)
  • Windows XP, Windows 7 (32-bit & 64-bit), Windows 8.1, Windows 10 (for running Visual Studio 6, 2008, 2010, 2012)
  • WIndows Server 2008 and 2012
  • FreeBSD 11-STABLE (for building FreeNAS)
  • Debian, Ubuntu, and CentOS Linux (for development; testing the Apache website and Postfix mailserver setups I run on DigitalOcean droplets; etc.)
And finally, there's a part of me that just hates to see computer resources sitting idle. The All-In-One (AIO) FreeNAS-as-a-VM-on-ESXi systems I use make better use of my server's resources than would a plain, bare-metal FreeNAS installation.
 

tvsjr

Guru
Joined
Aug 29, 2015
Messages
959
I'm a cybersecurity architect, so I have have a 3-node vSphere cluster at home as a lab environment. It also runs my entire home environment (multiple AD DCs, a full web stack, mail server, database, certificate authority, firewalls, etc.)
 

gpsguy

Active Member
Joined
Jan 22, 2012
Messages
4,472
I use it at work and in a lab environment at home. In the olden days at home I used removable hard disks. This is a lot easier to manage.
 

iHeartMacs

Explorer
Joined
Oct 5, 2017
Messages
56
FWIW, I'm a software developer, so I'm not a 'typical home user'...

I use ESXi for virtualization because it's mature, robust, reliable... and free for home use!

In my experience, FreeBSD jails are too lightweight for some of my use-cases, while bhyve is relatively new and unproven. With jails, you are restricted to FreeBSD software packages, which often lag behind the Linux versions; Plex is a good example.

So I use FreeNAS strictly as a rock-solid file server; at which it excels.

Ever notice how many threads here on the forum are posted by users having problem with jails or bhyve? There are quite a few! I seldom have problems with virtualization, because ESXi mostly 'Just Works'. :D

Some of the VMs I run that a home user might also use:
For development work I run these VMs:
  • Oracle database server (11g, 12c)
  • Windows XP, Windows 7 (32-bit & 64-bit), Windows 8.1, Windows 10 (for running Visual Studio 6, 2008, 2010, 2012)
  • WIndows Server 2008 and 2012
  • FreeBSD 11-STABLE (for building FreeNAS)
  • Debian, Ubuntu, and CentOS Linux (for development; testing the Apache website and Postfix mailserver setups I run on DigitalOcean droplets; etc.)
And finally, there's a part of me that just hates to see computer resources sitting idle. The All-In-One (AIO) FreeNAS-as-a-VM-on-ESXi systems I use make better use of my server's resources than would a plain, bare-metal FreeNAS installation.

I have a very similar build to your 'BANDIT all except the Dell Perc, NIC and fans. I'm new to ESXi and it sounds like you have accomplished what I want to achieve (many times over) I'd like to run a Freenas backup for windows machines and TimeMachine backups and I want to run Windows 7 for a Network Video Server. I have 12 6TB WD REDS and I'm wondering if I need to have more LSI 9210-8i for what I would like to do. I'm sure I'll need to bone up on how to manage the network traffic for the NVR and the Backups. Is that possible on one card? Thx for any advice.
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
I have a very similar build to your 'BANDIT all except the Dell Perc, NIC and fans. I'm new to ESXi and it sounds like you have accomplished what I want to achieve (many times over) I'd like to run a Freenas backup for windows machines and TimeMachine backups and I want to run Windows 7 for a Network Video Server. I have 12 6TB WD REDS and I'm wondering if I need to have more LSI 9210-8i for what I would like to do. I'm sure I'll need to bone up on how to manage the network traffic for the NVR and the Backups. Is that possible on one card? Thx for any advice.
It all depends on your backplane. In BANDIT I have a direct-attached backplane supporting 24 drives. There are 6 SAS connections on the backplane. So it takes 3 of the LSI cards to support 24 drives (because each card has 2 ports). If you have a backplane with an expander, then you only need a single LSI card to drive all 24 drives.

Search the forum and you'll find some informative tutorials on LSI adapters, backplanes, etc.

Good luck!
 

rungekutta

Contributor
Joined
May 11, 2016
Messages
146
I use ESXi for bunch of VMs (on diskless refurbished HP workstation with Xeon and lots of RAM) and FreeNAS on separate hardware (supermicro x11, intel i3, 32GB ECC RAM and 6x3TB WD Red) for all storage. The FreeNAS box serves ESXi via NFS as well as client computers via SMB. Works very well. I can’t understand why anyone would want to run VMs on FreeNAS which still seems to be experimental at best and far beyond its core value proposition (at least historically). ESXi on the other hand is free, industrially proven and extremely stable, not picky on hardware, and comes with way more features including various hardware pass-through options and management GUIs. Makes a lot more sense I think.
 

rungekutta

Contributor
Joined
May 11, 2016
Messages
146
Oh, and to answer the question. ;-) 1 VM for mail server (postfix and dovecot), one for web + web mail, one for nextCloud, a minecraft server, and a command-and-control VM to manage various batch jobs plus backups etc. All on Centos or Ubuntu. Oh, and a Windows Server 2016 in evaluation mode just because.
 
Last edited:

ViciousXUSMC

Dabbler
Joined
May 12, 2014
Messages
49
Currently using Freenas bare metal with 3 jails running super rock solid (manual setup not pre-configured plugins) and one Ubuntu VM also running rock solid.
I use ESXi at work and if I was to rebuild now, would go ahead and try to integrate it into my setup, simply because its an extra safety measure and extra flexibility.

An example is how right now the new 11.1RC is out and I see some users having problems. If I update and it breaks my stuff. I will have to find a way to repair it. If I had a virtual layer between, I am a snapshot away from fixing things.
 

Zredwire

Explorer
Joined
Nov 7, 2017
Messages
85
I use ESXi at home and FreeNAS on separate hardware. My FreeNAS box serves ESXi via NFS as well as client computers via SMB. I use VMWare at work and have been working with it for many years so I am very comfortable with it. Since ESXI and Freenas are both free it seems like the best of both worlds to me. As for what VM's I run, well I have:

Untangle Firewall
Security Camera Server
Media Center Server
Plex Server
HomeAutomation Server
and then a few VM's I use for learning things, like a Win 10, a Network Monitoring VM, a PFSense VM, etc.

I used to have the Camera server, Home Automation and Media Center on one bare metal box and had a bunch of problems. Now all is stable and if I have to work on one it does not effect the others. ESXI and Freenas are a great combination!
 
Last edited:

Xelas

Explorer
Joined
Sep 10, 2013
Messages
97
Similar to others here, I run various media server stuff on my AiO setup, but although my machine is an AiO, I don't host the VM's drives on my pool. For a smaller setup, it's actually much cheaper and simpler to just thrown an SSD into the server as a standalone ESXi datastore and use that to host the VM drives, and use the pool for file-sharing. Otherwise, you need to overengineer the pool config (with SLOG, faster drives, etc) to be able to handle iSCSI or NFS fast enough to work as system drives for multiple VMs. Heres my setup, with some detail:

HW:
Xeon E3-1245v3, 32 GB RAM, running on a SuperMicro X10SL7-F. That board has an on-board LSI adapter, so 6 x sata ports on the Intel chipset + 8 x ports on the LSI adapter gives 14 ports to play with, total, and the LSI adapter works perfectly in pass-through in ESXi. It also has 2 x Intel NICs and a separate management NIC. Very sweet board.

ESXi boots off of a USB drive. I clone it every once in a while for backup, but ESXi is SO easy to set up (< 1 hour), that it's not a big deal if that drive croaks (which hasn't happened in 4 + years)
6 x WD Red drives connected to the LSI's ports. This is my file storage pool. Not a fan of these - will move to WD RE drives or maybe HGST NAS drives. Reliability has been meh.
2 x SSD drives connected to the Intel Chipset ports. These are datastores within ESXi.
SSD#1, Samsung 840 Pro. Hosts all of the VM boot drives. Even though the VMs all share that drive, they are all scream along very quickly - Windows and Ubuntu VMs boot in under 10 seconds, and no hiccups during use.
SSD#2, an OEM LiteOn 256GB drive pulled form a laptop upgrade. Dedicated to Plex Metadata, NZB watch folder. That Plex folder can really explode in size, and consists of hundreds of thousands of tiny files, and it was soaking up more than double it's true size on disk due to allocation size issues.

If a VM needs more space than I can allocate in it's boot volume, I just point to a share on the pool, and offload the bigger stuff, but Windows 10 is perfectly happy with 50 GB, and Ubuntu with 20 GB each, so it's really not a big issue.

VMs:
FreeNAS
Windows 10 (media management and Plex - Plex will soon be moved to an Ubuntu instance),
Ubuntu 1 (various utilities, some media automation),
Ubuntu 2 (dedicated to Crashplan),
OpenVPN appliance - will be converted to a stand-alone Ubuntu with community OpenVPN when I have the time.

I have 12 GB RAM allocated to FreeNAS, the rest to the VMs. I started off with 16 GB dedicated to freeNAS, but I've found that 12 GB is perfectly OK as well in my setup, now that I moved all the VM stuff out of the pool.

I made sure that the boot order in ESXi has FreeNAS coming up first, and has a delay to make sure it boots completely before booting the other VMs. That way, all the shared drives are "online" in the VMs when they are up.
 

toadman

Guru
Joined
Jun 4, 2013
Messages
619
I'm curious as to how/why a home user would utilize ESXi. How do _you_ use ESXi and what benefits do you find useful?

Like the others, I use it because "it works," and it's free. (Though admittedly I'm running a vcsa instance off licensing I received for free.) I have a two host cluster with each node hosting multiple vms and either the primary fileserver vm (freenas) or the backup fileserver vm (also freenas). And I move VMs around as needed when doing maintenance, etc. That way no services go down, ever.

It's a rock solid platform. As such, I haven't had a reason to stop using it.
 

gpsguy

Active Member
Joined
Jan 22, 2012
Messages
4,472
But the features you tout, are not free by a long shot. I paid about $600, just to get the VCSA. But, that license doesn't get me vMotion.

Like the others, I use it because "it works," and it's free. (Though admittedly I'm running a vcsa instance off licensing I received for free.)
 

toadman

Guru
Joined
Jun 4, 2013
Messages
619
Yep, quite expensive normally. And the vcsa likes to use resources, even if it's not doing much of anything. If I didn't have essentially free licensing I would probably use proxmox or something similar.
 

tvsjr

Guru
Joined
Aug 29, 2015
Messages
959
Join VMUG, then get the EVALExperience license. $200/year gets you a 365-day non-prod license (and yes, you can swap the keys on your expiration date without destroying your cluster or taking anything down) for:
VMware vCenter Server v6.x Standard
VMware vSphere® ESXi Enterprise Plus with Operations Management™ (6 CPU licenses)
VMware vCloud Suite® Standard
VMware vRealize Operations™
VMware vRealize Log Insight™
VMware vRealize Operations for Horizon®
VMware Horizon® Advanced Edition
VMware vSAN™
VMware Workstation Pro 14
VMware Fusion Pro 10
VMware NSX Enterprise Edition (6 CPU licenses)
VMware vRealize Network Insight
VMware vRealize Automation 7.3 Enterprise

That's usually enough to keep most any virtualization nerd busy. :)
 

gpsguy

Active Member
Joined
Jan 22, 2012
Messages
4,472
As a VMUG member, I'm well aware of this and I've mentioned it here before. For a home lab, it's an excellent deal.

Join VMUG, then get the EVALExperience license
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,996
I use ESXi (No paid features) strickly for home use and run it on two computers, the main computer runs VMs for FreeNAS, Ubuntu, and Windoze 7 and 10. FreeNAS runs as a typical NAS plus it run Plex for now. The second computer run ESXi as well and I have three VMs; Sophos UTM, Ubuntu, and FreeNAS (backup files).

The only reason I started using ESXi is because I've been using VMWare Workstation for many years developing/testing software on my computer. I was very familiar with it and using ESXi was an easy move and this is probably the most mature and stable type 1 hypervisors available. What sucked was the lack of a nice full featured user interface, I do miss it but I don't miss it enough to pay $200/year for it. Yup, I'm a cheap b@stard.
 
Status
Not open for further replies.
Top