Shutdown script to send commands to ESXi via SSH

[Turgin]

TL;DR How can I send SSH commands to another system during the FreeNAS shutdown process?

Cyberpower CP1500AVRLCD via USB

I've come to the point in my FreeNAS/ESXi build of setting up my UPS. I have the UPS configured in FreeNAS and it will power down properly when I pull the power from my UPS. Moving on to the ESXi server I'd like to have all the powered-on guests be suspended, or failing that powered off, before the host shuts down and before FreeNAS shuts down.

I've done all sorts of Googling for a solution and I almost have it. I can feel it. Found a script here that with a little editing does exactly what I want. I have setup public/private key ssh login to ESXi and I can execute the script from a FreeNAS command line like so:

Code:

ssh root@192.168.40.31 /vmfs/volumes/nas01-iscsi/scripts/esxidown.sh

I think I'll have this solved if I can get that command into the FreeNAS shutdown process and thought that adding a command to "Tasks>Init/Shutdown Scripts" would get me what I want, but it doesn't seem to be working.

So, am I on the right track here? How can I get this command to execute when I need it to? Ideally, I'd like the FreeNAS shutdown to wait on the script to complete but with a maximum execution time so if the script hangs FreeNAS doesn't wait indefinitely to shutdown. The script itself has some basic checking so that it doesn't end up waiting forever for guests to suspend or power down.

Is there a better way? I've spent hours over the last two days trying all sorts of solutions found on the Google and haven't found a completely working one yet. Biggest problem with most of the other ways I've tried is that the free ESXi 6.0 doesn't seem to suspend guests when shutdown from CLI no matter how I have the Virtual Machine Startup/Shutdown properties configured. Yes, I have VMware Tools installed.

 

[fta]

Setting up a NUT server on your freenas box and then using an actual NUT client would be your best bet: http://translate.google.com/transla...2012/05/client-nut-pour-esxi-5-0/&prev=search

 

[Spearfoot]

TL;DR How can I send SSH commands to another system during the FreeNAS shutdown process?

Cyberpower CP1500AVRLCD via USB

I've come to the point in my FreeNAS/ESXi build of setting up my UPS. I have the UPS configured in FreeNAS and it will power down properly when I pull the power from my UPS. Moving on to the ESXi server I'd like to have all the powered-on guests be suspended, or failing that powered off, before the host shuts down and before FreeNAS shuts down.

I've done all sorts of Googling for a solution and I almost have it. I can feel it. Found a script here that with a little editing does exactly what I want. I have setup public/private key ssh login to ESXi and I can execute the script from a FreeNAS command line like so:

Code:

ssh root@192.168.40.31 /vmfs/volumes/nas01-iscsi/scripts/esxidown.sh

I think I'll have this solved if I can get that command into the FreeNAS shutdown process and thought that adding a command to "Tasks>Init/Shutdown Scripts" would get me what I want, but it doesn't seem to be working.

So, am I on the right track here? How can I get this command to execute when I need it to? Ideally, I'd like the FreeNAS shutdown to wait on the script to complete but with a maximum execution time so if the script hangs FreeNAS doesn't wait indefinitely to shutdown. The script itself has some basic checking so that it doesn't end up waiting forever for guests to suspend or power down.

Is there a better way? I've spent hours over the last two days trying all sorts of solutions found on the Google and haven't found a completely working one yet. Biggest problem with most of the other ways I've tried is that the free ESXi 6.0 doesn't seem to suspend guests when shutdown from CLI no matter how I have the Virtual Machine Startup/Shutdown properties configured. Yes, I have VMware Tools installed.

I've written some FreeNAS scripts to shut down VMs at system shutdown, borrowing heavily from the same source you cited. You can find the scripts at the bug report I mention in this post:

https://forums.freenas.org/index.php?threads/esxi-home-server-freenas.43527/page-2#post-287381

 

[Spearfoot]

Here's a direct link to the bug report I made earlier this month, and which contains the shutdown scripts I mentioned above.

You'll have to update to the latest 9.10-STABLE (FreeNAS-9.10-STABLE-201605240427) to use my scripts - or any other shutdown scripts that require networking; missing network support was the bug I reported. :)

https://bugs.freenas.org/issues/15323

 

[Turgin]

Thanks for the replies.

I do have NUT configured in FreeNAS and I found a link where someone created a NUT client plugin for ESXi which does work other than the guest suspend/shutdown problem I mentioned. The ESXi plugin will only initiate shutdown when battery is low. It would work well for me if I could have ESXi begin shutdown after a specified delay when power fails then have FreeNAS shutdown when battery power low. Alas, I could not arrive at that combination.

I did not think of a NUT client inside a guest to do the work. That's what I will try.

Edit: I just looked a the link provided and that is the plugin I used. It does work but with the caveats I mentioned.

 

[fta]

I'm pretty sure the "Shutdown timer" feature of the UPS settings does what you want. Set the freenas "Shutdown mode" to "UPS reaches low battery". When the UPS reaches low battery it will inform the nut client of it. The client will shut down your ESXi box. Then freenas will wait "Shutdown timer" seconds before shutting down. I have tested this on my setup (I use Xenserver rather than ESXi, though).

 

[Turgin]

I'm pretty sure the "Shutdown timer" feature of the UPS settings does what you want. Set the freenas "Shutdown mode" to "UPS reaches low battery". When the UPS reaches low battery it will inform the nut client of it. The client will shut down your ESXi box. Then freenas will wait "Shutdown timer" seconds before shutting down. I have tested this on my setup (I use Xenserver rather than ESXi, though).

/sigh
Sometimes I make things too difficult. Its a habit I think.

 

[Turgin]

I'm pretty sure the "Shutdown timer" feature of the UPS settings does what you want. Set the freenas "Shutdown mode" to "UPS reaches low battery". When the UPS reaches low battery it will inform the nut client of it. The client will shut down your ESXi box. Then freenas will wait "Shutdown timer" seconds before shutting down. I have tested this on my setup (I use Xenserver rather than ESXi, though).

So it wasn't this simple. "Shutdown timer" is only available when the mode is "UPS goes on battery". Since the NUT client for ESXi only operates in "UPS reaches low battery" mode I don't see how I can make it work this way.

I'll figure this out one way or another. Maybe NUT client in a guest and initiate the guest shutdown script from there or give that Cyberpower OVA a shot.

 

[fta]

So it wasn't this simple. "Shutdown timer" is only available when the mode is "UPS goes on battery".

That's not true. The "shutdown timer" parameter ends up as the FINALDELAY upsmon parameter. It waits FINALDELAY seconds after sending the client the NOTIFY_SHUTDOWN. If you have it set to "ups reaches low battery", that's when the NOTIFY_SHUTDOWN will be sent. See http://networkupstools.org/docs/man/upsmon.conf.html.

EDIT: You can verify this yourself by looking at the code that generates the upsmon config file: https://github.com/freenas/freenas/blob/9.10-STABLE/src/freenas/etc/ix.rc.d/ix-ups

 

[Turgin]

That's not true. The "shutdown timer" parameter ends up as the FINALDELAY upsmon parameter. It waits FINALDELAY seconds after sending the client the NOTIFY_SHUTDOWN. If you have it set to "ups reaches low battery", that's when the NOTIFY_SHUTDOWN will be sent. See http://networkupstools.org/docs/man/upsmon.conf.html.

EDIT: You can verify this yourself by looking at the code that generates the upsmon config file: https://github.com/freenas/freenas/blob/9.10-STABLE/src/freenas/etc/ix.rc.d/ix-ups

I'll verify when I get home after work but I am fairly certain the option grays out and won't accept input when you change the mode to "UPS reaches low battery". That indicated to me that it wasn't applicable to that mode but I didn't actually test it.

 

[Bidule0hm]

Yeah the option is grayed but the value is still used in the code, that's part of a bug (the delay is used twice leading to the system taking double the inputted delay to power off).

 

[Robert Smith]

Cyberpower also has a prepackaged CentOS based virtual machine (appliance) that works under ESXi and can be used for all kinds of UPS related tasks, including initiation of orderly shutdown.

 

[Turgin]

.. (the delay is used twice leading to the system taking double the inputted delay to power off).

That is some useful information right there.

Cyberpower also has a prepackaged CentOS based virtual machine (appliance) that works under ESXi and can be used for all kinds of UPS related tasks, including initiation of orderly shutdown.

I had seen that and it is on my list to try out as well, but my goal is to have FreeNAS be the "master". No reason other than that's the way I want it.

 

[Spearfoot]

That is some useful information right there.

I had seen that and it is on my list to try out as well, but my goal is to have FreeNAS be the "master". No reason other than that's the way I want it.

You could use something along the lines of this code as the last line of your FreeNAS shutdown task script:

Code:

ssh [userid]@[esxi_host] esxcli system shutdown poweroff -d 10 -r "UPS initiated shutdown"

When your UPS signals FreeNAS to power down, FreeNAS will execute its shutdown script and this snippet should power down the ESXi server. You may want to tweak the delay parameter to insure that FreeNAS has a enough time to finish shutting down.

I haven't tried it myself; my shutdown script just powers down all running VMs when FreeNAS goes down.

Reference: https://pubs.vmware.com/vsphere-51/...manage_hosts.4.2.html?path=2_1_0_1_0_0#992328