SOLVED HELP! - Replication to additional Drive(s) to be taken Offsite

[Thund3rDuck]

Hello, I am currently looking for some additional understanding in the replication tasks. How do I setup a task to replicate my main pool to an offsite drive. I am currently syncing to an onsite backup drive and it is working just fine. Now I want to get my data offsite on an encrypted drive to physically take to another location.

What is the best way to do this?

Server Specs:
HP Gen 7 N40L (BIOS Hack)
8GB ECC RAM
2 x WD 8TB White Label (Shucked from WD MyBooks)
1 x 4TB WD RED (NEW for onsite backups)
2 x 2TB (misc drives for offsite backup)
2 x 16GB USB 3.0 SanDisk Ultra Fit (mirrored boot)

Thank you for your help.

 

[cobrakiller58]

I don't think there is a best way that suits everyone so it depends on your situation. I have a case with free hotswap bays and access to secure storage but not a high speed network there, so I setup a replication task to the back-up pool I take offsite. When the replication is complete I disable the task, detach the pool, remove the disks and place them in a carrying case to take back to storage within 24 hours of the arriving onsite, recently acquired a BDXL burner and will be making more copies of the really important stuff that way. For my scenario at the moment that is the best for ME. IF I had access to a site that had high speed network access and stability I would setup another box there and have an offsite/online backup there AS WELL.
my previous case did not have any hotswap bays at all and zero free places to mount drives so it ended up NOT having a backup for a majority of the data because it was a pain to manage.

 

[Thund3rDuck]

I don't think there is a best way that suits everyone so it depends on your situation. I have a case with free hotswap bays and access to secure storage but not a high speed network there, so I setup a replication task to the back-up pool I take offsite. When the replication is complete I disable the task, detach the pool, remove the disks and place them in a carrying case to take back to storage within 24 hours of the arriving onsite, recently acquired a BDXL burner and will be making more copies of the really important stuff that way. For my scenario at the moment that is the best for ME. IF I had access to a site that had high speed network access and stability I would setup another box there and have an offsite/online backup there AS WELL.
my previous case did not have any hotswap bays at all and zero free places to mount drives so it ended up NOT having a backup for a majority of the data because it was a pain to manage.

Do you not have any other backup of your data? This seems risky to me. I couldn't do that with my data.

 

[Thund3rDuck]

I need an automated way to handle this. I would like to just setup another replication task to different drive and utilize the GUI tools provided. I'm currently logged in over ssh and zfs sending the last snapshot to another drive. I don't want to have to do this every time I want to update my offsite drive or rotate the drives out.

 

[cobrakiller58]

If you set up a server at another location you can set the task to replicate to it when ever you would like automatically, I am taking a risk but I do me best to minimize it with what I have to work with. You could also use one of the cloud providers and upload your data to their servers for a fee, some advertise private encryption but if you lose the key you lose the data on their server.

 

[Thund3rDuck]

Just created an additional replication task and it just says "not ran since boot."

 

[Redcoat]

Just created an additional replication task and it just says "not ran since boot."

Do you have a question?

 

[Thund3rDuck]

If you set up a server at another location you can set the task to replicate to it when ever you would like automatically, I am taking a risk but I do me best to minimize it with what I have to work with. You could also use one of the cloud providers and upload your data to their servers for a fee, some advertise private encryption but if you lose the key you lose the data on their server.

I appreciate it but another server is out of the question and really has nothing to do with my original request. I am looking to replicate to another drive or pool for offsite purposes. My setup is as follows:

1. zpool called MainVault consisting of 2 x 8TB drive in a mirror
2. zpool called OnsiteBackup4TB consisting of 1 x 4TB drive
3. zpool called OffsiteA2TB consisting of 1 x 2TB drive
4. zpool called OffsiteB2TB consisting of 1 x 2TB drive

MainVault has a recursive periodic snapshot task set to happen at midnight every day and a one year lifetime. I then have a replication task that is allowed to run all day and sends data from MainVault to OffsiteA2TB. This one is up to date and is working just fine. I setup another replication task is set send data from MainVault to another pool called OffsiteA2TB. This one just says, not ran since boot. What am I doing wrong here?

 

[Thund3rDuck]

Do you have a question?

Redcoat, the question is posted at the begining of this thread. I was posting an update as to what I was trying. How do I setup replication to multiple so that it actually sends data to those locations. My second replication task is just sitting there telling me it has not ran since boot. I must be missing some piece of the logic.

 

[Redcoat]

This one just says, not ran since boot. What am I doing wrong here?

Have you waited long enough for it to run?

Are you sure that the setup is 100% correct - no typos, full destination path specified, keys OK, etc.?

 

[Thund3rDuck]

Have you waited long enough for it to run?

Are you sure that the setup is 100% correct - no typos, full destination path specified, keys OK, etc.?

Yup, no errors at all. Nothing about the path being incorrect, issue with the keys or anything. Just a blank box under status and then under Last Run, it just says not ran since boot. I created an identical task and waited a full day with it still sitting there telling me it had not ran. As I understand it, the replication task should be taking the periodic snapshot and sending it to the locations I have setup. I even tried setting up an additional periodic snapshot to see if that helped and it still did not replicate to the Offsite Drive.

 

[Redcoat]

OK, it was the "Just created" in a new post that had me fooled on the wait time.

AFAIK what you are trying to do should work fine. The 2 (or now 3?) replication tasks should run serially once a new snapshot is created.

 

[Thund3rDuck]

OK, it was the "Just created" in a new post that had me fooled on the wait time.

AFAIK what you are trying to do should work fine. The 2 (or now 3?) replication tasks should run serially once a new snapshot is created.

I will wait again but I have already done this. Not sure how to kick start the process. Attached is a snapshot of the replication tasks menu in the modern UI.

 

[Thund3rDuck]

Well, i double checked this morning and still nothing I have a task setup for OffsiteA and OffsiteB but nothing happens. Is there something I’m missing? I attached a screenshot of the Replication task menu. The bottom two tasks are the offsite tasks. No errors, it just sits there.

 

[Redcoat]

1. zpool called MainVault consisting of 2 x 8TB drive in a mirror
2. zpool called OnsiteBackup4TB consisting of 1 x 4TB drive
3. zpool called OffsiteA2TB consisting of 1 x 2TB drive
4. zpool called OffsiteB2TB consisting of 1 x 2TB drive

To make sure I understand:

Replication to "2" is working, but replication to "3" and "4" are not working? And the pools "Offsite..." are pools on the same server with "Main Vault" and "Onsite...".

Please share the replication task configuration pages for 2 and either of 3 or 4? BTW, why are you blanking your internal IP addresses? And, presumably, all three blanks show the same server IP address if my scenario description above is correct?

 

[Thund3rDuck]

Afternoon,

Yes replication of my main pool is working just fine to the INTERNAL drive called OnsiteBackup4TB (#2 as you called it). Replication to (#3) is an internal 2TB drive named OffsiteA. Replication just sits there without any error and does not do anything. This is also the case for (#4) OffsiteB which is in a eSata conncted 2 Bay DAS case. Also I'm blanking out the DNS name of my FreeNAS box because I don't want to share any info I don't have to. All replication tasks are pointed to my FreeNAS server where the hard drives reside.

 

[Redcoat]

OK. Assuming that the target datasets exist, my guess is that the SSHKeys are the problem.

I had problems with this aspect when recommissioning my primary server after a motherboard change. I didn't get the copy/paste right and the desired replication of snapshots to my backup server didn't occur and I got no indication that it did not occur. This is at least part of the reason that I replied to your post.

I suggest that you retrace the keys step.

Thanks for the confirmation that the blanks are showing the same info. If I were really struggling I'd put the server's IP address in there "just in case"...
as I never tried replication to a DNS name and thus have no history on that. I assume it should be fine, but ...

 

[Thund3rDuck]

The target datasets do exist. If the setup is the same, i’m not understanding why the ssh keys would come into play. I let the task populate the information itself after clicking scan ssh key. No further input was done by me. Wouldn’t I get a failure status of some kind? I got one before when the path was off or the key was incorrect. This just doesn’t seem to make any sense.

Interesting note btw, if I click scan for ssh key in the modern UI, it is missing information. I have to go into the legacy UI and scan for the key. Seems to be a bug.

 

[Redcoat]

if I click scan for ssh key in the modern UI, it is missing informatio

That may have been the issue I experienced. I did not get any message indicating replication failure. As everything else in my config was ok I "redid" the keys, twice actually as I recall.

 

[Thund3rDuck]

That may have been the issue I experienced. I did not get any message indicating replication failure. As everything else in my config was ok I "redid" the keys, twice actually as I recall.

To test, I copied the key from the task that is working and pasted it into the task going to OffsiteA. I then changed the start time to 1am and the OffsiteB task to 2am. I normally don't make more than one change at a time for proper troubleshooting purposes but I figured those two changes would be harmless as I essentially have a one change control with OffsiteB (only changed the time it starts.)

While doing this it made me think. Could the periodic snapshot schedule and the 3 replication task schedules be causing some kind of conflict that the last two tasks just don't start? How should I set the schedule? I have researched for hours this weekend and can't seem to find the answers i'm looking for. I wish that the snapshot and replication portions of the guide went more in depth.