Resource icon

Utility scripts for FreeNAS and VMware ESXi

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Spearfoot submitted a new resource:

Utility scripts for FreeNAS and VMware ESXi - Scripts for use with FreeNAS-on-ESXi All-In-One systems

I've created a GitHub repository containing the scripts I've written for use with my FreeNAS-on-ESXi All-In-One systems:

https://github.com/Spearfoot/utility-scripts-for-freenas-and-vmware-esxi

These might be handy for folks who haven't already cooked up their own solutions to the standard issues:
  • Forcing ESXi to rescan datastores so it will see the ones provided by FreeNAS
  • Starting up selected VMs when FreeNAS boots
  • Shutting down VMs gracefully when FreeNAS is...

Read more about this resource...
 

listhor

Contributor
Joined
Mar 2, 2020
Messages
133
I can't get script to discover VMs on any of the datastores:
Code:
Thu Nov  5 17:17:35 CET 2020: Shut down virtual machines on ESXi host 172.16.0.8, FreeNAS server freenas
Thu Nov  5 17:17:35 CET 2020: /root/scripts/esxi/esxi-stop-all-datastore-vms.sh root@172.16.0.8 datastore=[STORE_ISCSI] (max wait time=180s
Full list of VM guests on this server:
Vmid     Name                       File                         Guest OS         Version                                                                                                                           Annotation                                                                                                                         
12     Raspbian    [Store_iscsi] Raspbian/Raspbian.vmx      debian10_64Guest      vmx-15                                                                                                                                                                                                                                                               
13     Ubuntu      [Store_nvme] Ubuntu/Ubuntu.vmx           ubuntu64Guest         vmx-18                                                                                                                                                                                                                                                               
16     win10       [Store_iscsi] win10/win10.vmx            windows9_64Guest      vmx-18   
17     CentOS      [Store_iscsi] CentOS/CentOS.vmx          centos8_64Guest       vmx-15                                                                                                                                                                                                     
18     Alpine      [Store_iscsi] Alpine/Alpine.vmx          other4xLinux64Guest   vmx-15                                                                                                                                                                                                     
19     Alpine2     [Store_iscsi] Alpine2/Alpine2.vmx        other4xLinux64Guest   vmx-15                                                                                                                                                                                                                                                               
23     win7        [Store_iscsi] win7/win7.vmx              windows7_64Guest      vmx-17                                                                                                                                                                                                                                                               
24     OPNsense2   [Store_nvmevo] OPNsense2/OPNsense2.vmx   freebsd12_64Guest     vmx-18                                                                                                                                                                                                                                                               
25     DappNode    [Store_nvmevo] DappNode/DappNode.vmx     debian10_64Guest      vmx-18                                                                                                                                                                                                                                                               
3      FreeNAS     [Store_nvme] FreeNAS/FreeNAS.vmx         freebsd11_64Guest     vmx-18                                                                                     
4      OPNsense    [Store_nvme] OPNsense/OPNsense.vmx       freebsd11_64Guest     vmx-18                                                                                                                                           
VM guests on datastore STORE_ISCSI:
Found 0 virtual machine guests on 172.16.0.8 datastore STORE_ISCSI
   Total shut down: 0
Total powered down: 0
Thu Nov  5 17:17:37 CET 2020: /root/scripts/esxi/esxi-stop-all-datastore-vms.sh completed

I've tried with upper and lower case of datastore name and it didn't change anything... I use Esxi 7.0.1, any ideas how to troubleshoot it?
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
I can't get script to discover VMs on any of the datastores:
Code:
Thu Nov  5 17:17:35 CET 2020: Shut down virtual machines on ESXi host 172.16.0.8, FreeNAS server freenas
Thu Nov  5 17:17:35 CET 2020: /root/scripts/esxi/esxi-stop-all-datastore-vms.sh root@172.16.0.8 datastore=[STORE_ISCSI] (max wait time=180s
Full list of VM guests on this server:
Vmid     Name                       File                         Guest OS         Version                                                                                                                           Annotation                                                                                                                        
12     Raspbian    [Store_iscsi] Raspbian/Raspbian.vmx      debian10_64Guest      vmx-15                                                                                                                                                                                                                                                              
13     Ubuntu      [Store_nvme] Ubuntu/Ubuntu.vmx           ubuntu64Guest         vmx-18                                                                                                                                                                                                                                                              
16     win10       [Store_iscsi] win10/win10.vmx            windows9_64Guest      vmx-18  
17     CentOS      [Store_iscsi] CentOS/CentOS.vmx          centos8_64Guest       vmx-15                                                                                                                                                                                                    
18     Alpine      [Store_iscsi] Alpine/Alpine.vmx          other4xLinux64Guest   vmx-15                                                                                                                                                                                                    
19     Alpine2     [Store_iscsi] Alpine2/Alpine2.vmx        other4xLinux64Guest   vmx-15                                                                                                                                                                                                                                                              
23     win7        [Store_iscsi] win7/win7.vmx              windows7_64Guest      vmx-17                                                                                                                                                                                                                                                              
24     OPNsense2   [Store_nvmevo] OPNsense2/OPNsense2.vmx   freebsd12_64Guest     vmx-18                                                                                                                                                                                                                                                              
25     DappNode    [Store_nvmevo] DappNode/DappNode.vmx     debian10_64Guest      vmx-18                                                                                                                                                                                                                                                              
3      FreeNAS     [Store_nvme] FreeNAS/FreeNAS.vmx         freebsd11_64Guest     vmx-18                                                                                    
4      OPNsense    [Store_nvme] OPNsense/OPNsense.vmx       freebsd11_64Guest     vmx-18                                                                                                                                          
VM guests on datastore STORE_ISCSI:
Found 0 virtual machine guests on 172.16.0.8 datastore STORE_ISCSI
   Total shut down: 0
Total powered down: 0
Thu Nov  5 17:17:37 CET 2020: /root/scripts/esxi/esxi-stop-all-datastore-vms.sh completed

I've tried with upper and lower case of datastore name and it didn't change anything... I use Esxi 7.0.1, any ideas how to troubleshoot it?
I am 99% certain this problem is due to a change in the behavior of ESXi 7.0 from previous versions. I could be wrong, but in my experience, versions 5.5-6.7 always use uppercase names for datastores. For this reason, and in order to prevent errors, I therefore wrote the script to force datastore names to uppercase. Now, it seems, I will have to modify the script so that it no longer does this.

I am on vacation at the moment, but I will look into this next week.

Thanks for bringing this issue to my attention.
 

listhor

Contributor
Joined
Mar 2, 2020
Messages
133
I am 99% certain this problem is due to a change in the behavior of ESXi 7.0 from previous versions. I could be wrong, but in my experience, versions 5.5-6.7 always use uppercase names for datastores. For this reason, and in order to prevent errors, I therefore wrote the script to force datastore names to uppercase. Now, it seems, I will have to modify the script so that it no longer does this.

I am on vacation at the moment, but I will look into this next week.

Thanks for bringing this issue to my attention.
I did try commenting out of these few lines of code responsible for changing datastore name to uppercase but it didn't help...
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
I did try commenting out of these few lines of code responsible for changing datastore name to uppercase but it didn't help...
You still need to assign the third command-line parameter to the esxidatastore variable, like this:
Code:
esxidatastore=$3

And the datastore name you pass must exactly match the case of the datastore name on your ESXi system.

I've updated the script on the GitHub repository:
 

listhor

Contributor
Joined
Mar 2, 2020
Messages
133
I've updated the script on the GitHub repository:
As a matter of curiosity but not only... :smile: Is there any reason why all of these scripts are in separate files? Wouldn't have been better to have scripts in one file, divided in sections and toggled them on/off by some boolean variable?
As it is now, at least to me, it's like "Inception" movie, I need to go deeper and deeper to proper script and have other preconfigured as well. And since they are in separate files it's easy to loose a track. Don't get me wrong, I'm grateful for your efforts, it's just a suggestion.

EDIT:
NFS datastores - can they be also safely disconnected, as same as iscsi one?
So far I've added to esxi-stop-all-vms.sh following:
Code:
ssh root@"${esxihost}" esxcli storage nfs41 remove -v Store_nfs

and to esxi-rescan-datastores.sh:
Code:
ssh root@"${esxihost}" esxcli storage nfs41 add -H 10.55.1.2 -s /mnt/tank/VM_nfs -v Store_nfs
 
Last edited:

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
As a matter of curiosity but not only... :smile: Is there any reason why all of these scripts are in separate files? Wouldn't have been better to have scripts in one file, divided in sections and toggled them on/off by some boolean variable?
As it is now, at least to me, it's like "Inception" movie, I need to go deeper and deeper to proper script and have other preconfigured as well. And since they are in separate files it's easy to loose a track. Don't get me wrong, I'm grateful for your efforts, it's just a suggestion.
I wrote them for use on my own systems and wasn't concerned about having multiple files at the time. You can always combine them into a single file, as you suggest, if that's what you prefer.

EDIT:
NFS datastores - can they be also safely disconnected, as same as iscsi one?
So far I've added to esxi-stop-all-vms.sh following:
Code:
ssh root@"${esxihost}" esxcli storage nfs41 remove -v Store_nfs

and to esxi-rescan-datastores.sh:
Code:
ssh root@"${esxihost}" esxcli storage nfs41 add -H 10.55.1.2 -s /mnt/tank/VM_nfs -v Store_nfs
I've never experimented with removing datastores as my only goal was to gracefully shut down any VMs stored on a particular all-in-one system's FreeNAS-based datastores. Of course you're welcome to modify the scripts to suit your needs.
 

listhor

Contributor
Joined
Mar 2, 2020
Messages
133
I've never experimented with removing datastores as my only goal was to gracefully shut down any VMs stored on a particular all-in-one system's FreeNAS-based datastores. Of course you're welcome to modify the scripts to suit your needs.
Unfortunately NFS datastore must be removed prior to restarting its source/Freenas. Otherwise you'll end up with inaccessible datastore and at the end you have use cli to get it removed from esxi...
Thus my question is whether it really needs to be removed and added again or there's some rescan command for NFS datastores?
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
Unfortunately NFS datastore must be removed prior to restarting its source/Freenas. Otherwise you'll end up with inaccessible datastore and at the end you have use cli to get it removed from esxi...
Thus my question is whether it really needs to be removed and added again or there's some rescan command for NFS datastores?
I've never had to remove NFS datastores. I use the esxi-rescan-datastores.sh script in my FreeNAS startup script. I provided an example of this in the GitHub repository.
 

listhor

Contributor
Joined
Mar 2, 2020
Messages
133
I've never had to remove NFS datastores. I use the esxi-rescan-datastores.sh script in my FreeNAS startup script. I provided an example of this in the GitHub repository.
And I also have been using esxi-rescan-datastores.sh and I keep using it (with added NFS commands)...
Does rescanning work with NFS (v. 4.1) datastores? In my case, after Freenas reboot, NFS datastore is visible but not accessible in Esxi. When I did manual (in webgui) rescanning, after Freenas reboot, I couldn't get NFS datastore back online neither delete it in webgui. I had to remove it in cli and the re-add it.
 

Spearfoot

He of the long foot
Moderator
Joined
May 13, 2015
Messages
2,478
And I also have been using esxi-rescan-datastores.sh and I keep using it (with added NFS commands)...
Does rescanning work with NFS (v. 4.1) datastores? In my case, after Freenas reboot, NFS datastore is visible but not accessible in Esxi. When I did manual (in webgui) rescanning, after Freenas reboot, I couldn't get NFS datastore back online neither delete it in webgui. I had to remove it in cli and the re-add it.
I haven't tested the combination of TrueNAS 12 under ESXi 7.0, so it may be that the datascore rescan - which has worked for all previous combinations of FreeNAS + ESXi - may no not work with that combination.

EDIT: I have a test/development system, but it's older and doesn't support ESXi 7.x, so I won't be able to test this problem any time soon.
 
Last edited:

obsidiangroup

Dabbler
Joined
Oct 4, 2022
Messages
19
I know this is an old thread, but wanted to say that I've successfully used these scripts with ESXi 7.x and ESXi 8.x with TrueNAS Scale 12.0 and works like a charm. My TrueNAS is virtualized, with an HBA and NIC passthrough to it. All my other ESXi machines use the TrueNAS storage for datastores via iSCSI and just works. Whenever the NAS and/or the ESXi host are rebooted, once the nas VM starts, my iSCSI datastores become available and the VMs that live on that storage can then be booted. I use iSCSI and vSAN. Just wanted to thank you for these scripts. Have been using them for years
 
Top