Instructions for setting up ZFS send and receive to remote FreeNAS box
Short background…I have been using a FreeNAS box at work to store our office documents and a folder of our ultrasound image files and reports. After an incident about a year ago when I had a bunch of disks on the FreeNAS box go down, I had to rebuild it. Fortunately, I had backups and so I could just transfer all the files back to the server using the Windows shares. I decided then to setup a local separate FreeNAS box cobbled together with an old server and some hard drives to make a backup machine. Using ZFS send and receive to this one was easy since it was on the same network. At the same time, I wanted a backup to a remote machine, which is my home FreeNAS box so that they were physically separated. That was where I was stumped for nearly two weeks trying to figure out the whole setup. I had searched all over and I could not find any clear instructions about how to setup the whole thing. After a lot of trial and error, I finally managed to get it figured out and so this tutorial is for anyone who might be facing a similar issue. I understand that for those networking wizards and gurus on this site, this is a no brainer, but there are a lot of noobs out there like me and maybe it would help them.
I use pfSense as the router/firewall both at work and at home and so the setup is for that setup. It would probably be similar for other firewalls as well. I also have a static WAN IP in the office but a dynamic ISP assigned one for my home. I had already setup an OpenVPN server in the office pfSense box for me and another staff member to be able to connect to the office and I have the clients in the PC at home. That comes in real handy to do the setup with the FreeNAS GUI since I can have them side by side on the browser. There are a lot of videos out there about how to setup a OpenVPN server in pfSense and I would especially refer you to Tom Lawrence of Lawrence Technology Systems for his videos on this.
Next, I setup a Dynamic DNS account for my home setup so that I could have a fixed address. I used the No-IP Free version. Very simple process and the website is very clear on how to set this up. Next in my home pfSense, I setup the Dynamic DNS service under the Services and they have the configurations for No-IP free there. You just need the user ID and password and you are set. Pfsense will now constantly update the IP address and resolve it to the Dynamic DNS service.
Next, setup NAT or port forward on pfSense. Again, a simple process, but I made sure to put in the source IP as the static IP of my office, resolving port 22, 80, 443 and 873 to the IP address of my home FreeNAS box. Even though ZFS send and receive is done using SSH on port 22, I put in the others, since I did see that it asked for port 80 and whether it resolved to HTTPS on the remote end. I put in 873 in case I decided to use rsync for some reason. Probably not necessary. I felt that since I had limited the source IP address, it was not available to anyone else. (I have since tested this from another network with no access).
Once these steps were completed, I connected to the office network over OpenVPN and with my home FreeNAS and the office FreeNAS GUI open side by side, I was able to set up a replication task on my office machine (source). At this point, the setup of the replication task is the same as if it were on the same network. I used the Semi-Automatic method, by copying the temporary authorization key from my home machine to the source or office machine. For the destination IP, just put in your Dynamic DNS address and voila!
After nearly two weeks of frustration and research, I was able to get the replication done without a hitch. Maybe it will save someone some time in this process. But, I did learn a lot and I guess that counts for something.
Short background…I have been using a FreeNAS box at work to store our office documents and a folder of our ultrasound image files and reports. After an incident about a year ago when I had a bunch of disks on the FreeNAS box go down, I had to rebuild it. Fortunately, I had backups and so I could just transfer all the files back to the server using the Windows shares. I decided then to setup a local separate FreeNAS box cobbled together with an old server and some hard drives to make a backup machine. Using ZFS send and receive to this one was easy since it was on the same network. At the same time, I wanted a backup to a remote machine, which is my home FreeNAS box so that they were physically separated. That was where I was stumped for nearly two weeks trying to figure out the whole setup. I had searched all over and I could not find any clear instructions about how to setup the whole thing. After a lot of trial and error, I finally managed to get it figured out and so this tutorial is for anyone who might be facing a similar issue. I understand that for those networking wizards and gurus on this site, this is a no brainer, but there are a lot of noobs out there like me and maybe it would help them.
I use pfSense as the router/firewall both at work and at home and so the setup is for that setup. It would probably be similar for other firewalls as well. I also have a static WAN IP in the office but a dynamic ISP assigned one for my home. I had already setup an OpenVPN server in the office pfSense box for me and another staff member to be able to connect to the office and I have the clients in the PC at home. That comes in real handy to do the setup with the FreeNAS GUI since I can have them side by side on the browser. There are a lot of videos out there about how to setup a OpenVPN server in pfSense and I would especially refer you to Tom Lawrence of Lawrence Technology Systems for his videos on this.
Next, I setup a Dynamic DNS account for my home setup so that I could have a fixed address. I used the No-IP Free version. Very simple process and the website is very clear on how to set this up. Next in my home pfSense, I setup the Dynamic DNS service under the Services and they have the configurations for No-IP free there. You just need the user ID and password and you are set. Pfsense will now constantly update the IP address and resolve it to the Dynamic DNS service.
Next, setup NAT or port forward on pfSense. Again, a simple process, but I made sure to put in the source IP as the static IP of my office, resolving port 22, 80, 443 and 873 to the IP address of my home FreeNAS box. Even though ZFS send and receive is done using SSH on port 22, I put in the others, since I did see that it asked for port 80 and whether it resolved to HTTPS on the remote end. I put in 873 in case I decided to use rsync for some reason. Probably not necessary. I felt that since I had limited the source IP address, it was not available to anyone else. (I have since tested this from another network with no access).
Once these steps were completed, I connected to the office network over OpenVPN and with my home FreeNAS and the office FreeNAS GUI open side by side, I was able to set up a replication task on my office machine (source). At this point, the setup of the replication task is the same as if it were on the same network. I used the Semi-Automatic method, by copying the temporary authorization key from my home machine to the source or office machine. For the destination IP, just put in your Dynamic DNS address and voila!
After nearly two weeks of frustration and research, I was able to get the replication done without a hitch. Maybe it will save someone some time in this process. But, I did learn a lot and I guess that counts for something.
