ESXi NFS Datastore - What are Best Practices?

[Xelas]

great ... converted to iSCSI ... left server rebooting overnight, the iSCSI datastore never refreshed even after hours. only way is to "rescan" volumes manually after boot.

i was having the same issue. My all-in-one system worked beautifully in esxi 5.0, ok-ish in 5.5, but 6.0 seems to hate not seeing the iscsi "drives" and doesn't handle that very well. I ended up moving my vm images to a separate drive (an ssd) outside of FreeNAS, and redesigned my VMs so that they have smaller system drives, but store most of their data on network shares. Esxi all-in-ones are a PITA.

This bypasses the whole chicken and egg thing with esxi not seeing iscsi drives upon bootup and failing to bit my vms. I've tried playing with delays, scripts, then just decided that my time and sanity are worth more than the cost of an ssd.

 

[wreedps]

What version of 6? I am running ESXi6 U1 and my FreeNas connected via ISCSI flies!

 

[depasseg]

I think the issue was if freenas is virtualized on the ESXi6 host, then ESX won't see the FreeNAS iscsi share at boot and doesn't reconnect easily later once FreeNAS has finally booted.

 

[wreedps]

I have 5 ESXi 6 U1 hosts at my home connected to a 9.3 Freenas over 10gbe ISCSI and my performance is phenomenal.

something else is a factor......

 

[depasseg]

Performance isn't the issue. The issue is that FreeNAS is running as a VM on a single ESX host. When the ESX host boots, FreeNAS isn't running yet. Once the FreeNAS VM is up, running, and serving NFS or ISCSI shares, the ESX host isn't automatically connecting so those datastores remain offline.

 

[wreedps]

ohhh thats sounds like trouble. I am out... Sorry.

 

[Xelas]

Yep. It works, but is way more trouble than its worth. A cheap ssd (in my case, a 256 GB Samsung 840 pro) can easily hold 4 VMs with excellent performance, so I just have all vm images on that. My FreeNAS instance has a dedicated intel nic and lsi adapter, both in vt-d passthrough, so performance is great.
I really don't have enough VMs to justify a whole separate box.

 

[wreedps]

Run it the right way and maybe there wont be trouble?

If not get you a cheap 4 bay NAS that does ISCSI. You will be happy. I do testing of numerous apps at a time, why I need all the power. I have 100+ VMs online at my house right now. When I need them or my team, they perform.

 

[reqlez]

Performance isn't the issue. The issue is that FreeNAS is running as a VM on a single ESX host. When the ESX host boots, FreeNAS isn't running yet. Once the FreeNAS VM is up, running, and serving NFS or ISCSI shares, the ESX host isn't automatically connecting so those datastores remain offline.

By the way guys ... yes it can be trouble, but after fine tuning things ( took a while to get it working properly ) it is very stable.

Ben wrote a guide, and then we figured out a few "gotchas" with NFS with ESXi6 ... basically have to run a script at startup inside freenas to "reboot" NFS service on start up. Also, you have to set at least 2 minutes or more "delay" before you execute your next VM that is on the NFS datastore freenas provides. And also a "shutdown" delay for freenas VM is also a good idea ... so that you don't shut down freenas before your other VM shuts down. But ... depending on the situation, this CAN be problematic if you don't tune it properly. All i can say, proceed with caution and do lots of testing and research.

guide and comments are here: https://b3n.org/freenas-9-3-on-vmware-esxi-6-0-guide/

Oh by the way ... not using iSCSI anymore with this single all in one setup at least, had iscsi connection drops ... now on NFS ... no issues ... but u need a good SLOG like S3700