automatic backup server

[VioletDragon]

Hi folks,

I am building a backup server however I want to have automatic backups from my main server which is also running FreeNAS however I want the backup server to be off then wakes up via WOL (Wake On LAN). Wakes up to do a backup then enters sleep again. I have been looking at Rsync Tasks. Any recommendations?

Thanks.

Jack.

 

[Chris Moore]

automotive backups

Does that mean that you want the backup server to be in your car? It is a bit confusing.
You can setup an rsync between the two FreeNAS systems and schedule it to start at a specific time. You would need to let that run a few times while monitoring the system to see how long it takes. The first one will be the longest and the duration of the backup will depend on the amount of data that needs to transfer.
There is no need to do wake on LAN because you will have to schedule the start time of the transfer on the main server, you might as well just set a power on time in the BIOS / UEFI of the backup server. Again, you need to time how long it takes for the boot process to complete so you can have the backup server turn itself on early enough that it is fully started and ready when the main server initiates the backup or the backup will fail.
Having already timed how long a backup takes, you can setup a cron job on the backup server to have it issue a shutdown command at a certain time after the backup should have completed. Automation of all of these steps is very simple.
The most complicated is the rsync so here is a guide:
http://thesolving.com/storage/how-to-sync-two-freenas-storage-using-rsync/
The guide is doing a two way sync, but you only need a one way sync, even easier.
This can also be done using ZFS send/receive, but you asked about rsync specifically.
There is also a feature of ZFS where you can take snapshots of the filesystem state. These snapshots can be accessed without damaging the current state and I have used that to "step back in time" and retrieve an older copy of a file that had later been changed or deleted. It is a handy feature that eliminates some of the normal reasons for having a backup. A backup is still good for disaster recovery though.
Especially if it is in your car, so you can take it with you.

 

[VioletDragon]

It depends what the best thing to do is. Manually or automated. The machine is not going to be on 24/7 and want to keep backup of my data.

Thanks for your reply I will take a look. I am guessing that the Backup Server will be the Server under Rsync?

 

[Chris Moore]

Thanks for your reply i will take a look. Im guessing that the Backup Server will be the Server under Rsync?

The rsync module in FreeNAS requires some configuration on each side of the link. I have used it for years and I find it better to do a push from the machine with the data to the machine where you want the data. A pull configuration is much easier to do wrong.

 

[VioletDragon]

The rsync module in FreeNAS requires some configuration on each side of the link. I have used it for years and I find it better to do a push from the machine with the data to the machine where you want the data. A pull configuration is much easier to do wrong.

So im guessing that Server 1 will be push and Server 2 will be pull if im understanding this correctly?

 

[Chris Moore]

and Server 2 will be pull if im understanding this correctly?

You need not pull. A push from server 1 is all that is needed. You just need to setup server 2 to be receptive to the push.

 

[diskdiddler]

So do all modern bios support an automatic turn on timer from the menu?

If not. What other solution could he use?
I know you could literally use a timer which turns the power point on and off, the on part is fine, it's getting the system to be in a state that's still powered on, yet receptive to bring powered off without data loss (think: the old days "it's now safe to turn off your PC, before acpi)

 

[Chris Moore]

So do all modern bios support an automatic turn on timer from the menu?

You bring up a good question. I made the assumption that they do, because I deal with many that do, but I went looking and found that not all system firmware supports that feature.
You can still send the wake on lan command, I suppose.

 

[VioletDragon]

You need not pull. A push from server 1 is all that is needed. You just need to setup server 2 to be receptive to the push.

Should i do the first backup manually? or let FreeNAS do it for me?

 

[VioletDragon]

So do all modern bios support an automatic turn on timer from the menu?

If not. What other solution could he use?
I know you could literally use a timer which turns the power point on and off, the on part is fine, it's getting the system to be in a state that's still powered on, yet receptive to bring powered off without data loss (think: the old days "it's now safe to turn off your PC, before acpi)

The motherboard I am using does have the option in the BIOS. However my other Intel Desktop Board does not have this option but I have already tested it and works fine.

 

[Chris Moore]

For me, because of the amount of data to transfer, I connected all the drives to one system and did a copy from one pool to the other.
Once a pool is created on a set of disks, those disks can be moved to another system and the pool imported.
After the initial copy was done, which went very fast since the drives were all in the same system, I exported the pool, moved it back to the other system and keep it synchronised using rsync. Transfering all that data over the network would have taken more than a week, but it finished in just over one day doing it locally.

 

[VioletDragon]

Okay if that is the case. I will copy everything from my main server to my Backup server then set up rSync. One more question though. Does the pool have to be same name as my main server? or can it be a different name? Thanks.

 

[Chris Moore]

Mine are named different from each other.
When they were in the same server, they had to be different. It is only a matter of setting up your path property.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[VioletDragon]

One thing I am not too sure on is that the Automatic Shutdown in FreeNAS. What would I need to do? What command do I need to use? And can I shut it down after so many hours? i.e an hour or 2 hours? I have got the BIOS timing sorted.

 

[Chris Moore]

One thing im not too sure on is that the Automatic Shutdown in FreeNAS. What would i need to do? What Command do i need to use? and can i shut it down after so many hours? i.e an hour or 2 hours? I've got the BIOS timing sorted.

You just want to use the BIOS to turn the system on, not off. The command for system shutdown in unix (which FreeBSD is and FreeNAS is built from FreeBSD) is this: shutdown -P now It will bring down the system immediately, and automatically power off the system. When that command is issued, anything that is running will be stopped. It is issued from the command line, but you can put it in a cron schedule so that you can have it happen at a specified time. To figure out when you need to do that, you will have to run the rsync task a few times to see how long it usually takes to complete the transfer. You can monitor the CPU activity on the two systems to see when the rsync is running. It will start low, when the files are being compared, then jump up when it is actually transferring files and drop back down when it is done. When my system is running the transfer it will average around 20% CPU usage on the sending system and around 10% on the receiving system. Once the CPU utilization drops to a low level, you probably want to give it a few extra minutes to ensure everything is finished and then the system can be shutdown. Anything that is not done can be copied on the next sync. After the first sync, which can take days over the network, the updates can be done in as little as half an hour, depending on how much you have changed. You will want to figure an average for how long they take and give it a little extra time, then schedule the shutdown accordingly. This may take a week or more to get figured.

 

[VioletDragon]

You just want to use the BIOS to turn the system on, not off. The command for system shutdown in unix (which FreeBSD is and FreeNAS is built from FreeBSD) is this: shutdown -P now It will bring down the system immediately, and automatically power off the system. When that command is issued, anything that is running will be stopped. It is issued from the command line, but you can put it in a cron schedule so that you can have it happen at a specified time. To figure out when you need to do that, you will have to run the rsync task a few times to see how long it usually takes to complete the transfer. You can monitor the CPU activity on the two systems to see when the rsync is running. It will start low, when the files are being compared, then jump up when it is actually transferring files and drop back down when it is done. When my system is running the transfer it will average around 20% CPU usage on the sending system and around 10% on the receiving system. Once the CPU utilization drops to a low level, you probably want to give it a few extra minutes to ensure everything is finished and then the system can be shutdown. Anything that is not done can be copied on the next sync. After the first sync, which can take days over the network, the updates can be done in as little as half an hour, depending on how much you have changed. You will want to figure an average for how long they take and give it a little extra time, then schedule the shutdown accordingly. This may take a week or more to get figured.

Have i done this correctly? I cant seem to get it working.
Server 1

What about Server 2?

 

[VioletDragon]

Rsync doesnt work. Ive spent so many hours now trying different thing but doesnt work. Everytime i try something different i keep getting this error any ideas?

Feb 28 10:49:00 vaultmort rsync: @Error: Unknown module 'sync'
Feb 28 10:49:00 vaultmort rsync: rsync error: error starting client-server protocol (code 5) at main.c(1662) [sender=3.1.2]

 

[Chris Moore]

Rsync doesnt work. I've spent so many hours now trying different thing but doesnt work. Everytime i try something different i keep getting this error any ideas?

Feb 28 10:49:00 vaultmort rsync: @Error: Unknown module 'sync'
Feb 28 10:49:00 vaultmort rsync: rsync error: error starting client-server protocol (code 5) at main.c(1662) [sender=3.1.2]

I gave you a web link to the "how to" for the setup. You might have missed a step. The rsync module is configured on the target system so it knows where to put the data that is sent from the source system.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[VioletDragon]

I gave you a web link to the "how to" for the setup. You might have missed a step. The rsync module is configured on the target system so it knows where to put the data that is sent from the source system.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

Yes but doesnt work. Ive tried the guide you linked me all i get is this.

Feb 28 10:49:00 vaultmort cron[6907]: _secure_path: /nonexistent/.login_conf is not owned by uid 1000
Feb 28 10:49:00 vaultmort rsync: @Error: Unknown module 'sync'
Feb 28 10:49:00 vaultmort rsync: rsync error: error starting client-server protocol (code 5) at main.c(1662) [sender=3.1.2]

 

[VioletDragon]

Update. Ive decided to use replication task with ZFS on my backup Server.