FreeNAS on ESXi serving VMs via NFS

[Spearfoot]

So I rebooted my host system today, and had it set to boot my FreeNAS VM and no other. The script was set to rescan my datastores, but when it was all booted, I still couldn't see my NFS datastores, just show up as emptly. I manually ran the command

Code:

ssh <username>@<esx mgmt IP> esxcli storage core adapter rescan –all

from the FreeNAS Shell (FreeNAS 11.2U5) and I substituted root for the username and the proper IP for my Host ESXi install, but still no datastore. I had to manually remove and add it to get it to work. Does this command no longer force the rescan of NFS drives, or is there something else missing?

Try running your command with two dashes in front of the 'all' option, like this:

Code:

esxcli storage core adapter rescan --all

As for the applicability of this rescan command for NFS datastores... I dunno. And I certainly bow to @jgreco's superior knowledge and experience.

I wrote this several years ago, when I was using both block (iSCSI) and NFS datastores. I only use NFS datastores now, but nevertheless it still seems to work, and it works on all three of my FreeNAS servers.

 

[Newfoundland.Republic]

Try running your command with two dashes in front of the 'all' option, like this:

Code:

esxcli storage core adapter rescan --all

As for the applicability of this rescan command for NFS datastores... I dunno. And I certainly bow to @jgreco's superior knowledge and experience.

I wrote this several years ago, when I was using both block (iSCSI) and NFS datastores. I only use NFS datastores now, but nevertheless it still seems to work, and it works on all three of my FreeNAS servers.

@Spearfoot - I am in the process of building my FreeNAS server and was wondering about NFS v. iSCSI. I have iSCSI running on an old Synology DS211j and it works fine for my home lab. However, it seems that iSCSI on FreeNAS has a whole bunch of considerations as compared to NFS.

Why did you switch to NFS?

Thanks!

 

[jgreco]

I'm not @Spearfoot but I will chime in with the observation that NFS has the attractive property of working without a ton of fuss. iSCSI has a lot of features going for it, but it is difficult to do well and do correctly. If you want things like redundant paths and load balancing, and don't mind your data being locked up in the VMFS file system format, iSCSI may be a good choice, but the first time you run into a situation where you lose your partition table or need to resignature the thing, you will be questioning your choices.

 

[Newfoundland.Republic]

Thanks @jgreco - I going back to NFS. I used to use in on my old Synology DS211j that was just sharing space for my ESX box (home lab). Great info to consider!

 

[Spearfoot]

@Spearfoot - I am in the process of building my FreeNAS server and was wondering about NFS v. iSCSI. I have iSCSI running on an old Synology DS211j and it works fine for my home lab. However, it seems that iSCSI on FreeNAS has a whole bunch of considerations as compared to NFS.

Why did you switch to NFS?

Thanks!

Mostly because it's just easier, as alluded to by @jgreco. If I were configuring a production system I wouldn't use NFS, but for a simple home lab/single-user virtualized environment like mine, it works just fine.

If you're curious, there's a great deal of well-informed opinion about iSCSI and NFS here on the forum.

 

[Terydan]

Try running your command with two dashes in front of the 'all' option, like this:Code:
esxcli storage core adapter rescan --all

Your command is the correct one, that is the only way it will take the command. I just mistyped in my posting, I double checked it. The description for rescan is

Code:

Rescan SCSI HBAs to search for new Devices, remove
DEAD paths and update path state. This operation will
also run an claim operation equivalent to the
claimrule run command and a filesystem rescan.

which makes me dubious of its ability to rescan NFS Stores and feels like it's focus is only SCSI/iSCSI. I am interested in the command that jgreco mentioned

Code:

esxcfg-nas -r

but I couldn't find anything online about it other than a reference to it being for INTERNAL USE ONLY. I might just have to try it out. I did see a posting that all "esxcfg" commands are being deprecated in favor of the "esxcli" commands, but saw some postings of how sometimes the esxcli commands can cause more damage than their older counterparts, specifically a reference to disruption of permissions.

EDIT: ssh'd into my ESXI host and ran "esxcfg-nas --help" and it states that the "-r" flag is to "Restore all NFS version 3 mounts from the configuration file. (FOR INTERNAL USE ONLY)". So it looks like it won't even work for nfs41 mounts.

 

[Spearfoot]

which makes me dubious of its ability to rescan NFS Stores and feels like it's focus is only SCSI/iSCSI.

Based on the command summary, I can understand why you might feel this way. But I'm telling you that I've used it successfully to rescan datastores on three servers, and for several years.

Note that the VMware documentation also states that, in addtion to adding iSCSI adapters, it also performs a filesystem scan, as noted below, and perhaps this is what 'wakes up' NFS datastores:

Code:

--skip-fs-scan | -F By default after all rescan operations a filesystem scan is
performed to add newly found filesystems and remove any filesystems that are
no longer available. Passing this flag will skip that filesystem scan. 

 

[Terydan]

I honestly am not sure what is going on for me then. I just powered everything down to replace my UPS unit on the server so I had a chance to play around with the shell and I just tried that command again with absolutely zero results.

BUT... I believe I have found a partial solution to this problem (at least in my case). I just started using the "--help" feature of the shell and slowly peeled back the layers of the "esxcli storage core adapter rescan --all" command to see all the different options of each part of the command. I finally found using

Code:

ssh <username>@<esxi_host_ip> esxcli storage filesystem mount -l <nfs_datastore_name>

successfully mounts my NFS Datastore.

Now my only problem is that my VM needs to be reconfigured before it can boot because for some reason even though it is not set to autostart, it detects that the second hard drive - which points to NFS datastore - is not present and defaults to a dummy 32GB hard drive that still references the NFS datastore that was not mounted during host boot. Is there some somewhere I can set a delay for the VM to stop it from searching for its resources? Once it resets that hard drive in the VM settings it makes me delete the second hard drive and re-add the hard drive selecting existing and choosing the vmdk from the NFS datastore manually.

Edit: Input this new command into my post init task for FreeNAS, but it doesn't work because according to FreeNAS the post init takes place before services start, therefore the NFS service isn't working yet and the rescan can't find it. I need to find a way to perform the command after services have started.