Shared iSCSI connection and ESXi 6.7U3

[veldthui]

I currently have a Synology NAS and have a shared iSCSI drive that is shared between 2 hosts. These hosts are not in a cluster. Currently it works perfectly and I can stick a VM on either host and have them on the iSCSI datastore. I can then move the VM's if the host they are on needs to be shut down.

I now have a FreeNAS 11.3RC1 setup and want to do the same thing there. I have done some searches and some posts says it is not possible for two hosts to share the same iSCSI datastore the same way Synology does and other posts suggest it will work but don't say how.

I saw one suggestion that said to simply point each host to the same LUN and it would work.

Can someone please confirm if FreeNAS can do share an iSCSI datastore with more than one eSXI host that are not in a cluster. If they can what steps do I need.

Thanks.

 

[veldthui]

Okay so no replies so thought I would just do some tests.
Set up Freenas to has a single Initiator/Target/Extent to a 500GB zvol.
I then set up the first esxi host and it found the volume and asked me how I wanted to format it. Formatted as VFS6.
Did the same on the second esxi host and it found the volume on the rescan and just added it to the host. Did not ask if I wanted to format it.
Same with the 3rd host.

Appears to be accessible from hosts no trouble and can write to it from each host. Not going to keep anything important on it for now and will continue testing writing to it from the 3 hosts.

I know that Synology has a checkbox allowing multiple initiators and it would not allow me to connect a second host until it was ticked so maybe Freenas allows multiple initiators to the same one.

 

[mav@]

FreeNAS does not limit multiple initiators access to the same LUN, since SCSI protocols do not require that. It is up to admin to not shoot its foot by using non-clustered file systems from multiple initiators.

 

[jgreco]

FreeNAS does not limit multiple initiators access to the same LUN, since SCSI protocols do not require that. It is up to admin to not shoot its foot by using non-clustered file systems from multiple initiators.

Points awarded for unhelpful answer. ;-)

Can someone please confirm if FreeNAS can do share an iSCSI datastore with more than one eSXI host that are not in a cluster. If they can what steps do I need.

ESXi's VMFS5 and VMFS6 are cluster filesystems.

You cannot do the stupid things with iSCSI. Or, rather, you can, and no one stops you. The stupid things include trying to hook up two Windows boxes to a single iSCSI disk and run NTFS. NTFS is not a cluster-aware filesystem, and each PC will make writes to the iSCSI disk that quickly end up corrupting the filesystem. This is what @mav@ was talking about without actually explaining it. I do not doubt you have read about iSCSI badness from rage-gamers who hooked two Windows PC's up to their big iSCSI gaming share that they did because some games refuse to work against a NAS share. And it worked. For days or weeks. Until one day they had both PC's running. Then FreeNAS gets trash-talked for ruining their massive carefully set up gaming "share." Because NTFS isn't designed for that. FreeNAS actually did exactly what it was supposed to... and shared the block storage device... and we can't be too angry with @mav@ about his answer since he literally wrote the current iSCSI implementation. :^)

On the flip side, VMFS5 and VMFS6 are cluster filesystems. The instant you hook more than one ESXi host to an iSCSI volume, you have a cluster. It does not matter that you seem to think you don't. "cluster" does not mean "managed by vSphere" in this context. It means a set of computing systems that work cooperatively in some manner. The manner is that they correctly share the block storage device. This is just a basic requirement for large-scale virtualization.

So if you read your sentence I quoted above, it literally makes no sense. You either hook up one ESXi host to the datastore (in which case it ain't a cluster) or you hook up multiple ESXI hosts to the datastore (in which case it is a cluster). Either thing will work. The reason it works is because VMFS is a cluster-aware filesystem and correctly makes sure that operations are done in a safe manner -- it is assuming multiple hosts are accessing the datastore.

So you're fine. Just do it.

 

[veldthui]

Points awarded for unhelpful answer. ;-)



ESXi's VMFS5 and VMFS6 are cluster filesystems.

You cannot do the stupid things with iSCSI. Or, rather, you can, and no one stops you. The stupid things include trying to hook up two Windows boxes to a single iSCSI disk and run NTFS. NTFS is not a cluster-aware filesystem, and each PC will make writes to the iSCSI disk that quickly end up corrupting the filesystem. This is what @mav@ was talking about without actually explaining it. I do not doubt you have read about iSCSI badness from rage-gamers who hooked two Windows PC's up to their big iSCSI gaming share that they did because some games refuse to work against a NAS share. And it worked. For days or weeks. Until one day they had both PC's running. Then FreeNAS gets trash-talked for ruining their massive carefully set up gaming "share." Because NTFS isn't designed for that. FreeNAS actually did exactly what it was supposed to... and shared the block storage device... and we can't be too angry with @mav@ about his answer since he literally wrote the current iSCSI implementation. :^)

On the flip side, VMFS5 and VMFS6 are cluster filesystems. The instant you hook more than one ESXi host to an iSCSI volume, you have a cluster. It does not matter that you seem to think you don't. "cluster" does not mean "managed by vSphere" in this context. It means a set of computing systems that work cooperatively in some manner. The manner is that they correctly share the block storage device. This is just a basic requirement for large-scale virtualization.

So if you read your sentence I quoted above, it literally makes no sense. You either hook up one ESXi host to the datastore (in which case it ain't a cluster) or you hook up multiple ESXI hosts to the datastore (in which case it is a cluster). Either thing will work. The reason it works is because VMFS is a cluster-aware filesystem and correctly makes sure that operations are done in a safe manner -- it is assuming multiple hosts are accessing the datastore.

So you're fine. Just do it.

Thanks. I was aware VMFS6 was cluster aware and that it would work if the system allowed it. I thought it would be okay one the first one asked if I wanted to format the datastore and the other did not ask.
I was just about to reply to @mav that VMFS6 was cluster aware so that his reply did not apply to this.

As I said it will be just used as non important data for some time to try it out. Also wanting to test if it is fast enough for vMotion.

 

[jgreco]

Thanks. I was aware VMFS6 was cluster aware and that it would work if the system allowed it. I thought it would be okay one the first one asked if I wanted to format the datastore and the other did not ask.

Right, because you wouldn't want to format a datastore already formatted and mounted on a different hypervisor, now, would you?

 

[HoneyBadger]

I have done some searches and some posts says it is not possible for two hosts to share the same iSCSI datastore the same way Synology does

Hopefully those posts weren't here on the FreeNAS forum, because as you've discovered, that's quite a bit of misinformation.

 

[jgreco]

Hopefully those posts weren't here on the FreeNAS forum, because as you've discovered, that's quite a bit of misinformation.

Well, I don't know. I like to come down on these as I see them, but I've been caught off guard a number of times by posts I simply missed.