Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

Local backup hdd to hdd w/o raid

Western Digital Drives - The Preferred Drives of FreeNAS and TrueNAS CORE
Status
Not open for further replies.
Joined
Apr 8, 2013
Messages
39
Hi,

I would like to setup two harddrives to backup (w/o increments) two other harddrives without linking them with RAID.

So here is what I want:
HDD w/Jail --> HDD1Backup
HDD w/misc Storage --> HHD2Backup

I wanted to do this with Rsync tool bundled into Freenas 8 but it seems to only support Freenas --> External distant storage. So I went with a cron job with excludes of mnt, sys and tmp but exlude does not work, so all mnt fs are copied in the process.

I wonder if there's a simple way to make local HDD to HDD backups on Freenas 8, or if I have to figure a DIY way ? If rsync w/cron is the only option, what would be the best rsync command to copy everything without increments and without dynamic folders like mnt or sys or tmp ?
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
Rsync should work just fine. Use local host as the remote connection.

Sent from my Galaxy Nexus
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
I havent tried this yet, as I havent had time, but

under Services>Rsync>Rsync Modules, add a module to point to the location you want to rsync FROM (your data you want to backup)

then go back to System>Rsync Tasks
Path: the spare drive
Add Rsync Task and remote host will be localhost
Rsync Module
Rsync Module Name: whatever you gave your module
Pull
set your time/dates/options/etc

try that and see if it works for you

(technically you could reverse those as well: Module point to the TO location, Path: data to be backed up, Push)
 
Joined
Apr 8, 2013
Messages
39
Thank you so much Pirateghost :D

It works flawlessly, and is much more easy to setup and reliable than a cron job with custom rsync command ! Exactly what I was looking for.
 
Joined
Apr 8, 2013
Messages
39
Btw, there's still one thing that bothers me: /mnt folder is still backed up in the process, so every mounted pools in the jail are rsync'd as well. Is there a way to avoid that ?
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
Why are you rsyncing all of /mnt?
Only sync the directories you need. Have multiple jobs set up if you need.

Sent from my Galaxy Nexus
 
Joined
Apr 8, 2013
Messages
39
So you think it's a bad idea to backup the whole HDD excluding what I don't need and that I should instead only backup say /etc and /usr for instance ?
 
Joined
Apr 8, 2013
Messages
39
Yes, the idea here is that I have spent days to setup my entire system complete with DNS/DHCP server, Mail server with roundcube, Owncloud, internet gateway with domain name etc... and I don't want to lose all my work because the HDD fails at some point.

So I want to backup my configs (and subsequent configs) and be able to restore them if needed. Hence the need for the /etc and /usr folders.

I don't want to backup any movies, music or anything alike.
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
Am I misunderstanding something? what /etc /usr folders? in your Jails?
 
Joined
Apr 8, 2013
Messages
39
Yes, because all of this has been setup inside a jail for security reasons. So I want to backup my work (mostly files in /etc and /usr from the jail).
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
Sorry, I don't use jails (I have a ProxMox cluster for all my VM needs).
It was my assumption that a jail is just a directory located on one of your storage disks, are you saying it mounts the disks at /mnt also? so the jail mounts the same storage its already mounted and running from?
 
Joined
Apr 8, 2013
Messages
39
Not at all. The Jail only mounts the pools I want it to mount (for DLNA purpose for instance) and those pools are not on the same Harddrives.

Essentially the jail is a bsd sub-system that acts as if it was the system itself. So you have Freenas on one hand (on usb stick), the different pools for different purposes (storage, backup...), and the jails for pbi's and every services that are not handled by Freenas (dns/dhcp, mail server, cloud backend accessible from the internet etc...).

In the best case, Freenas would handle everything by itself, but since it lacks some features, I like to have the power of it coupled with a complete freebsd system on its side (handled by the jail).

If you could explain me what a PoxMox cluster is, it would enlighten me for the night ;)
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
so jails are essentially like an openvz container?

I have my proxmox cluster setup with VMs to run my network.
torrent vm, newsgroup vm, webservers, jenkins build servers, backuppc server, game servers, etc.

anything that needs to talk to my FreeNAS boxes gets NFS linked to them. (Plex, Download machine, etc)

http://www.proxmox.com/proxmox-ve/features
 
Joined
Apr 8, 2013
Messages
39
Mmmh, I see. So you have multiple servers for each need I assume.

The jails are more like chroot's rather than vm's. There is no virtualization since this is just a sub-system with its own root and IP address. You can have as many jails as you want, so for instance 5 webservers on a single server (although you'd need a strong machine).

Proxmox seem interesting with its web interface, and powerful. Although a bit overkill for my needs it seems^^ Freenas + Owncloud and some extras are enough for me.

The interesting thing with the jail and its freebsd architecture, is that you can do virtually anything that a unix system can do. Whereas Freenas and Proxmox are limited with the functionnalities that the devs put into them. The downside is that you have to setup everything by hand, with command lines and all (almost always).

To go back into topic, I finally went with the solution above: I'm backing up /usr and /etc so that my config files are safe. The rest can be clean installed if needed, it won't do much harm. I'll just have to re-install the programs if there is any issue with the drive.
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
have a look at openVZ....its not a VM in the traditional sense. it sounds pretty much identical to a jail.

multiple servers for each need? yeah, just openvz containers or KVM/QEMU vm as needed. I used to put everything on one box, an all around everything server. it got annoying when i needed to bring down my entire system for one piece that was giving me fits. i have 5 physical servers here in my house (2 FreeNAS boxes, 3 Dell R710's [2 of which are in a ProxMox cluster, 1 is an ESXi host])

I guess I dont understand your comment about how ProxMox is limited. I dont want my hypervisor/host to do much more than one function so that I can let the VMs and containers do their jobs. Thats kind of the point of virtualization in general. i dont like the idea of running anything other than storage services on my storage nodes, i guess its just the fact that i have been burnt in the past by having everything in the same box.

i have considered setting up a spare machine with FreeBSD and learning the jails system....one day, until then, I am too busy with what i already have.
 
Joined
Apr 8, 2013
Messages
39
one day, until then, I am too busy with what i already have.
:D I believe you! It's a nice setup you've got, and I think you're right: one server should serve one purpose. But I'm just entering the game and didn't have that much money to spend, so I ended up with one server only that handle storage and cloud backend alike.

I don't really know if ProxMox is limited, I understood that you had only like 50 'appliances' that you could install. But now that I look deeper, it seems that you can have any vm that you want. Although I would assume that you need a strong horsepower to have them run at decent speed.

But why do you need vm's for instance ? I understand that with containers you can have torrent, dlna etc..., but what are the os's vm's for ?
 

pirateghost

Unintelligible Geek
Joined
Feb 29, 2012
Messages
4,219
:D I believe you! It's a nice setup you've got, and I think you're right: one server should serve one purpose. But I'm just entering the game and didn't have that much money to spend, so I ended up with one server only that handle storage and cloud backend alike.

I don't really know if ProxMox is limited, I understood that you had only like 50 'appliances' that you could install. But now that I look deeper, it seems that you can have any vm that you want. Although I would assume that you need a strong horsepower to have them run at decent speed.

But why do you need vm's for instance ? I understand that with containers you can have torrent, dlna etc..., but what are the os's vm's for ?
For whatever I want. :smile:

I run all kinds of operating systems in my VM environment. Testing, building, playing.

Sent from my Galaxy Nexus
 
Status
Not open for further replies.
Top