Sharing data on iSCSI targets remotley or locally..Please Read

[Chett]

Let me explain a bit as to what we have been doing.

We have a remote office, we are all MACs in our firm. every night we backup from a mac server to a mounted iSCSI target on a remote mac.
This has been fine. But from what I read, sharing will be a problem as from what I gather multiple people cannot and should not access a remote or local iSCSI targets because it can result in data corruption. Here is the scenario in short. If there is a catastrophic problem within the office we will require multiple people to vpn in to my mac that is remote to my office and access the backed up data there. They will need read and write access on the remote end on mounted sharepoints (network shares) connected to the data on the iSCSI target. (as having everyone access the iSCSI target directly is not good...from what I read)
Can I share over SMB or AFP? Or is this still an invalid configuration. Unless WebDAV may work? They will need to be able to save documents through WebDAV if that is possible.

I am not sure/clear on the proper procedure to do this. Any help would be greatly appreciated.

 

[jgreco]

The problem with iSCSI is that it is a block level protocol.

Good search term: "cluster aware filesystem"

https://forums.freenas.org/index.php?threads/two-servers-connecting-to-iscsi-target.35124/

So when an operating system mounts a hard disk, it generally caches useful things in memory and assumes that no one else is writing to the hard disk. You can substitute "SATA disk" or "USB disk" or "USB flash" in there if those feel more comfortable to the discussion. A typical filesystem that is NOT cluster-aware makes this assumption.

Imagine simultaneously plugging a single SATA disk or USB flash device into two PC's. This is what iSCSI effectively makes possible. Now you have a situation where two PC's may be trying to access the same data. Device one reads a file named "abc" and happens to read in a directory block and caches it. Device two wants to write "def". Reads the same directory block. Finds the slot right after "abc" is open, so writes a directory entry there, now the physical disk block contains directory entries for "abc", "def". Device one now decides to write "ghi". It has the cached block, sees "abc", sees the empty entry right after it, so writes "ghi" there. Now the physical disk block contains "abc", "ghi". And we've lost information. What happened to "def"? And the data that the second PC is writing to that file?

Cluster-aware filesystems allow multiple hosts to simultaneously mount a single disk. This is possible because the cluster-aware filesystems take extra steps to signal between nodes the writes that are being staged out, and also they tend to do much less read caching. VMFS is one of the best examples of this.

Your typical consumer grade filesystems, NTFS, Apple, etc., are optimized for speed, and try to maximize caching, and are NOT cluster-aware. They will happily trample your disk and can corrupt everything if you get multiple mounts going.

So you absolutely need to use AFP, CIFS, NFS, or other file-level sharing protocols for multi-host access in the situation you describe. It does not matter if the backing disk is connected via SATA, USB, SAS, iSCSI, Fiber Channel, etc. -- as long as a single host has the filesystem's disk mounted, and is responsible for coordinating the integrity of the filesystem, you'll be fine. This is why most network storage works with these protocols. You only have one kernel accessing and one filesystem mount active, which means that there's no chance of inadvertent trampling.

You were on the right track with your design. Hopefully this helps to explain the WHY. I am sorry if this was either too wordy or not wordy enough. I want to make sure you get the idea.

 

[Chett]

The problem with iSCSI is that it is a block level protocol.

Good search term: "cluster aware filesystem"

https://forums.freenas.org/index.php?threads/two-servers-connecting-to-iscsi-target.35124/

So when an operating system mounts a hard disk, it generally caches useful things in memory and assumes that no one else is writing to the hard disk. You can substitute "SATA disk" or "USB disk" or "USB flash" in there if those feel more comfortable to the discussion. A typical filesystem that is NOT cluster-aware makes this assumption.

Imagine simultaneously plugging a single SATA disk or USB flash device into two PC's. This is what iSCSI effectively makes possible. Now you have a situation where two PC's may be trying to access the same data. Device one reads a file named "abc" and happens to read in a directory block and caches it. Device two wants to write "def". Reads the same directory block. Finds the slot right after "abc" is open, so writes a directory entry there, now the physical disk block contains directory entries for "abc", "def". Device one now decides to write "ghi". It has the cached block, sees "abc", sees the empty entry right after it, so writes "ghi" there. Now the physical disk block contains "abc", "ghi". And we've lost information. What happened to "def"? And the data that the second PC is writing to that file?

Cluster-aware filesystems allow multiple hosts to simultaneously mount a single disk. This is possible because the cluster-aware filesystems take extra steps to signal between nodes the writes that are being staged out, and also they tend to do much less read caching. VMFS is one of the best examples of this.

Your typical consumer grade filesystems, NTFS, Apple, etc., are optimized for speed, and try to maximize caching, and are NOT cluster-aware. They will happily trample your disk and can corrupt everything if you get multiple mounts going.

So you absolutely need to use AFP, CIFS, NFS, or other file-level sharing protocols for multi-host access in the situation you describe. It does not matter if the backing disk is connected via SATA, USB, SAS, iSCSI, Fiber Channel, etc. -- as long as a single host has the filesystem's disk mounted, and is responsible for coordinating the integrity of the filesystem, you'll be fine. This is why most network storage works with these protocols. You only have one kernel accessing and one filesystem mount active, which means that there's no chance of inadvertent trampling.

You were on the right track with your design. Hopefully this helps to explain the WHY. I am sorry if this was either too wordy or not wordy enough. I want to make sure you get the idea.

Thanks so much for responding, this was very helpful. I do have a remaining question that maybe I did not get from your reply.
Can I still share out an iSCSI target remotely to multiple users? or Share it locally so when they vpn in they can map to the share using smb or some other protocol?

Thanks

 

[jgreco]

Only one operating system kernel should be attempting to mount and interpret the disk as a file system at any one time.

User A mounts iSCSI X, accesses some file, dismounts X. User B mounts iSCSI X, accesses some file, dismounts X. This sequence of events is fine.

User A mounts iSCSI X, accesses some file. User B mounts iSCSI X, accesses some file. Both then dismount X. This is not fine, because user A and user B's kernels both had the filesystem mounted at the same time.