FreeNAS + VMWARE + APC

[andycpp]

Hi,

Would you please share the best practice how to achieve power outage protected system for the following landscape:
1) I have 3 hardware units - FreeNAS 9.10.1 box, VMWare ESXi 6.x box and APC UPS with AP9630 network card
2) All VMWare VMs are located on FreeNAS via iSCSI
3) VMWare has special VM (vMA) with APC Network Shutdown installed (located on FreeNAS now too)
4) FreeNAS shares some disks via SMB with Active Directory authentication (Directory configured to sync with AD located on one of the VMs)

Before FreeNAS (VMs were stored on VMWare box's HDDs) VMWare works as needed - on power failure Network Shutdown received the signal from APC, shut down all VMs and turn off ESXi itself, on mains return - ESXi started automatically (BIOS power on AC return) and started all VMs (via ESXi feature)

Now I need to configure the same functionality taking FreeNAS into account. My idea is the following:
1) Move vMA to VMWare box HDDs
2) Configure FreeNAS to autostart on mains return (via BIOS)
3) Configure VMWare to autostart on mains return (via BIOA)
Startup phase
4) VMWare autostarts vMA only
5) in vMA:
4.1) Check if FreeNAS is up and running and iSCSI target is ready
4.2) Check VMWare iSCSI storage accessebility
4.3) Start VMs
4.4) Restart FreeNAS SMB (or Directory sync?) after AD VM is up and running (via ssh)
Shutdown phase
4.3) in APC Newtork Shutdown configure to run the script on shutdown signal (from APC)
4.4) in the shutdown script - shutdown VMs, shutdown FreeNAS (via ssh) and shutdown ESXi host

I'm not sure if this idea is the best one, so would you please suggest better ways to achieve power outage protected system. Also it would be great if you share some vMA scripts to do all these / or proposed steps :)

 

[dlavigne]

Did you decide upon this solution?

 

[bigphil]

I do a similar thing for my home servers. Your workflow looks good to me. I don't use vMA, but a Windows 7 box to run my scripts. Any newer version of Windows would work as well. On this pc, I've installed the VMware powerCLI tools and have the UPS plugged directly into the ESXi box via USB cable. I then pass the UPS USB device directly to the Windows 7 VM and it sees it as a battery. I just let Windows discover the device and install the built-in drivers for it. As long as it detects it as a battery, you're good. After that, I have a scheduled task that runs every couple of minutes to check the status.

Code:

Add-PSSnapin -Name 'VMware.VimAutomation.Core'

function SendMail {
Param([string]$MsgSubject,
	  [string]$MsgBody)

Send-MailMessage -Body $MsgBody -Subject $MsgSubject -From "UPS-Monitor@yourdomain.com" -Port 25 -Priority High -SmtpServer "YourSMTPserver" -To "YourEmailAddress"
} #end function SendMail

$BatteryInfo = Get-WmiObject win32_battery
[int]$BatteryStatus = $BatteryInfo.BatteryStatus

   If ($BatteryInfo.estimatedChargeRemaining -lt 50)
		{
	 [string]$ShutdownSubject = "SHUTTING DOWN SERVERS DUE TO POWER OUTAGE!"
	 [string]$ShutdownBody = "SHUTTING DOWN SERVERS DUE TO POWER OUTAGE!"
		 SendMail $ShutdownSubject $ShutdownBody
		 Connect-VIServer -Server YourEsxiHost -User YourDomain\YourUser -Password YourPasswordHere
		 Stop-VMHost -VMHost 'YourEsxiHost' -Force -RunAsync -Confirm:$False
		}

   If ($BatteryInfo.BatteryStatus -ne 2)
		{
	 [string]$StatusSubject = "Power event on the server UPS!"
	 [string]$StatusBody = "The battery status is not 2 (meaning no access to AC).`r`n`r`nBatteryStatus: $Batterystatus`r`nSee Microsoft article for status numbers.`r`n`http://msdn.microsoft.com/en-us/library/aa394074(v=vs.85).aspx"
		 SendMail $statusSubject $StatusBody
		}

As long as you have the VM start/stop settings configured for "Shutdown Guest OS", the host should shutdown the VM's gracefully. The script emails me if the VM detects that it is no longer on utility power, and only starts the shutdown process if the battery falls below 50%. As for shutting down FreeNAS during all of this, you could connect to the FreeNAS box via SSH if you install the Posh-SSH module for powershell. Tons of examples out there for using that module to connect to an SSH server.

For your startup phase, I'd do the following:
4.1 - Check that FreeNAS is running. No need to check the target. You could almost omit this entire part and run 4.2 only.
4.2 - On the Windows VM (If you insist on using the vMA, I'm sure you could do the same using "esxcli storage..." commands, but I think powershell is much better here), use the Get-Datastore cmdlet to query the "State" value of the datastore you wish to monitor. You'd know that FreeNAS is up and the iSCSI target is available when the value of the datastore state = "Available" and you could then proceed to start the rest of the VM's. If the value is anything but available, quit the script and run again in 1 minute. If you have multiple ESXi hosts in a cluster, be sure to set the proper rules to make sure this VM stays on the host with the UPS attached. Pretty easy to setup.