Two FreeNAS machines - ZVOL over FC

[X7JAY7X]

I have two FreeNas machines setup, lets call them FreeNAS1 and FreeNAS2. FreeNas1 is the iSCSI target that is setup with a ZVOL extent. It doesn't appear to be working or perhaps I don't know where the ZVOL should show up on the FreeNAS2 machine. If I create a disk extent on FreeNAS1, I can see it show in in the disks under storage on FreeNAS2. What am I doing wrong here with the ZVOL extent? Any help would be greatly appreciated.

 

[pirateghost]

You shouldn't be attempting a setup like this.

 

[X7JAY7X]

Why not? How should it be done? I am looking for high speed transfer between two NAS. Whats the best way to accomplish this?

 

[pirateghost]

Zfs send/receive.

You shouldn't be mounting iscsi on FreeNAS to another FreeNAS.

 

[X7JAY7X]

Thanks. How should I do this? I am trying to stick with FC if possible. If I cant use FC, other less desirable option would be 1g copper.

 

[pirateghost]

I don't understand what you're trying to accomplish. If you're mounting an iscsi target from one FreeNAS to another, that zvol is only accessible from that one FreeNAS. What is the point?

Step back.
Explain your setup.
Explain what you want to accomplish.
Be as clear as possible because it sounds like you aren't really clear.

 

[X7JAY7X]

Here is my environment. I have two FreeNAS machines. One is sharing a file extent over FC to 10 ESXi machines. Lets call this FreeNASESXi. FreeNASESXi has roughly 36TB and its almost used up by Virtual Machine files. I have another FreeNAS machine with 180TB of space. Lets call this FreeNASBackup. We need to archive some VM files. We want to transfer the VM files from FreeNASESXi to FreeNASBackup and then delete them off of FreeNASESXi. Theoretically, I would like to also be able to copy them back from FreeNASBackup to FreeNASESXi in case that VM is needed again, although, this is less important right now. I was looking to transfer between the two machines over 8GB FC because I already have the cards and FC switches for this. I also want to stick with FC (if possible) because of the transfer speed. If you think FC is not the way to go period, that is fine, I will have to revert to 1gb copper or potentially 4gb aggregated copper. Let me know what you think or if you need more information.

 

[pirateghost]

So please explain to me how you would use iscsi in this backup method of yours.

You do NOT want to attempt to attach to an iscsi target that's being presented by FreeNAS on another FreeNAS. That's not what you want.

I would probably use replication of the datasets, but if you want to delete the original, then you're looking more at rsync or scp.

 

[darkwarrior]

Here is my environment. I have two FreeNAS machines. One is sharing a file extent over FC to 10 ESXi machines. Lets call this FreeNASESXi. FreeNASESXi has roughly 36TB and its almost used up by Virtual Machine files. I have another FreeNAS machine with 180TB of space. Lets call this FreeNASBackup. We need to archive some VM files. We want to transfer the VM files from FreeNASESXi to FreeNASBackup and then delete them off of FreeNASESXi. Theoretically, I would like to also be able to copy them back from FreeNASBackup to FreeNASESXi in case that VM is needed again, although, this is less important right now. I was looking to transfer between the two machines over 8GB FC because I already have the cards and FC switches for this. I also want to stick with FC (if possible) because of the transfer speed. If you think FC is not the way to go period, that is fine, I will have to revert to 1gb copper or potentially 4gb aggregated copper. Let me know what you think or if you need more information.

Hi there,
if your only need is to move VMs from your primary "FreeNASESXi" server to the "FreeNASBackUP" machine, I would do the following which is much more straight forward:
Additionally you should be able to benefit from your 8Gbit FC connections
1. create a new ESXi datastore on the FreeNASBackUP machine
2. Connect your ESXi hosts to it
3. Use the Vmware functionality that is actually meeting all your needs: I present you Storage vMotion
3.1 Read up on that ;)
4. Move back and forth all the VMs between the different DataStores (as you see fit)
5. Enjoy the show
6. ...
7. profit

Have fun :D

 

[X7JAY7X]

darkwarrior, thanks for an FC solution. We have done this exact setup in the past, however, we are looking to cut out the intermediate ESXi to do the transferring. Plus, we do not have licenses for vMotion. I was wondering if there is another NAS to NAS solution over FC directly. I am not getting a clear answer on this up above. Rsync and SCP are network only (correct me if I am wrong). Perhaps there is no good FC solution to do this?

 

[toadman]

The problem you have with ISCSI is the volume (zvol) is formatted in vmfs. So freenas isn't going to be able to mount and read it no matter what you try. So you basically have two options I think:

(1) snapshot the zvol and replicate (zfs send/receive) from freenasesxi to freenasbackup as mentioned above. The issues with this are (a) it's gonna send the entire zvol, not individual vm folders/files and (b) to copy back you will have to mount the replicated zvol on freenasbackup as a new datastore on your esxi hosts so you can get to the individual vm files. But of course that would violate your issue of wanting to not use esxi for doing the actual copying. (OR, I suppose copy back the entire zvol from freenasbackup to freenasexsi and remount the datastore on freenasesxi, but that seems like a total waste vs just mounting it from freenasbackup.)

(2) setup a new datastore on freenasbackup. from there you can copy (or move) selected vms from the esxi host command line (no storage vmotion requred, but you'll have to do it with vms powered off) from the datastore on freenasesxi to the datastore on freenasbackup. Since you said above, "we are looking to cut out the intermediate ESXi to do the transferring," then I guess this option is not palatable for you. So you are left with option 1.

I suppose another option is to switch to using NFS datastores, then you don't have the iscsi issues and you could use replication, scripting, or whatever you want from freenas to freenas because then Freenas could actually see the vm files.

 

[toadman]

One more option (though it's manual in the free version) would be to use a solution like veeam to backup the VMs to a share on freenasbackup.

 

[X7JAY7X]

toadman, I was thinking of installing vmfs-tools to read the extent, however, this is a lot more involved than I first though. I guess, for now, I will continue to use the intermediate ESXi machine to copy files to/from the two NASs. I will revisit this later when I have more time. Thanks all for your help.

 

[toadman]

Yea, vmfs-tools seems like a pain. I think someone tried it on freenas in a jail and didn't get very far. Good luck on whatever route you take going forward. Using the esxi host seems easiest, but I can understand why you wouldn't want to do it that way.

 

[darkwarrior]

Indeed using Veeam Backup to save the VMs, offload them to the Backup, delete the original files and restore them when necessary could be another way to go. Even if it's not as elegant as a Storage vMotion :p