FreeNAS Backups/ RSYNC

[Aidan]

I have a small FreeNAS (11.3-U4.1) set up doing fileserving for a school. I’m trying to provide some level of protection from failure of the main box, so I am looking for advice on the best way to back up the files.

I have created a second (less powerful, less storage) box running the same version of FreeNAS. I plan to put it in a different location within the campus (on the same network).

So,

Do I set up the second unit with the same pools/ shares etc?

Do I populate the shares with a copy of the existing data from the network first, then use RSYNC (or whatever) to keep it refreshed? I can do this before I bring the unit to the school and if it would speed things up, it’d be good.

I also plan to set up snapshots.

Is this the right approach? Access to the school will be infrequent so a low-maintenance set up is best. Any advice appreciated. Thanks.

 

[anodos]

I have a small FreeNAS (11.3-U4.1) set up doing fileserving for a school. I’m trying to provide some level of protection from failure of the main box, so I am looking for advice on the best way to back up the files.

I have created a second (less powerful, less storage) box running the same version of FreeNAS. I plan to put it in a different location within the campus (on the same network).

So,

Do I set up the second unit with the same pools/ shares etc?

Do I populate the shares with a copy of the existing data from the network first, then use RSYNC (or whatever) to keep it refreshed? I can do this before I bring the unit to the school and if it would speed things up, it’d be good.

I also plan to set up snapshots.

Is this the right approach? Access to the school will be infrequent so a low-maintenance set up is best. Any advice appreciated. Thanks.

If possible you should use ZFS replication and not rsync.

 

[firsway]

I have a small FreeNAS (11.3-U4.1) set up doing fileserving for a school. I’m trying to provide some level of protection from failure of the main box, so I am looking for advice on the best way to back up the files.

I have created a second (less powerful, less storage) box running the same version of FreeNAS. I plan to put it in a different location within the campus (on the same network).

So,

Do I set up the second unit with the same pools/ shares etc?

Do I populate the shares with a copy of the existing data from the network first, then use RSYNC (or whatever) to keep it refreshed? I can do this before I bring the unit to the school and if it would speed things up, it’d be good.

I also plan to set up snapshots.

Is this the right approach? Access to the school will be infrequent so a low-maintenance set up is best. Any advice appreciated. Thanks.

If you're going to set up snapshots anyhow, then you might as well (as Anodos says) create replication tasks and just send those snapshots to your backup machine. They will automatically apply at the other end. As the filesystem is COW, then this will also cater for your initial sync (but initial will take longer)
I'd get the machine set up in its new location first, on the same network, so that when you do the initial sync, it will be using the LAN, and so shouldn't take as long as if you tried this over a remote link

 

[Constantin]

Two FreeNAS boxes allowing replication via ZFS is likely the gold standard - efficient, quick, error-free. It can even be done to an off-site location, further helping retain data.

Rsync to DAS backup arrays (something I do at home) is possible, though can be very time-intensive, especially if you are hosting many small files, use a 1GbE connection and a separate computer to host the DAS, and/or do not have a fast metadata L2ARC. Rsync really benefits from L2ARC in my testing - though one reason is that I am using a separate computer for hosting the RAID array as opposed to attaching it to the NAS directly.

I have since started compacting dormant data sets into ZIP archives or Apple Disk images to present ZFS with fewer, but larger files to help rsync performance. For example, a backup of an old system of mine had over 500k files in it, mostly really small stuff. Putting it into 8MB bands of a sparsebundle or a ZIP file allows rsync to process a lot more data in less time because there are fewer files to checksum and HDDs are not as efficient dealing with many small files.

I am now creating all of these archives to eliminate as many small files as I can before transitioning the server to 12.x, enabling special VDEVs that will contain the metadata and small files while also opening up the L2ARC to all files, not just metadata as presently. Key thing with special VDEVs being that you have to have the same redundancy there as you do with the rest of the pool or the whole pool will go kaplooie if you have a complete special VDEV failure (I have three mirrored S3710 for that purpose...)

So, if you can, try to limit the number of files you need to rsync, use a metadata L2ARC if you are wedded to rsync, etc. but consider the use of ZFS replication if the data is important and you have the budget.

 

[kiriak]

what about snapshot replication to DAS?

I'm new in FreeNAS and at the moment I only have an old PC setup as FreeNAS for test purposes and trying to decide to leave Synology or not.
I use a few snapshot replication tasks to an external USB disk.

Isn't it the next best solution for backup (besides replication to another ZFS server)
given the advantages of snapshots and ZFS, compared to the rsync solution?

Or am I missing sth?

 

[Constantin]

Not at all as long as you are ok with having the data in a ZFS format.

I happen to prefer having my DAS’ use a format that my home computer can read and write to natively in case I ever need to access the data at the offsite location, etc.

IIRC, @jgreco and others have pointed to issues with some eSATA port multipliers in external DAS enclosures not playing nice. FWIW, I haven’t had those issues (I use a hardware RAID array with an eSATA or USB connection).

 

[jgreco]

IIRC, @jgreco and others have pointed to issues with some eSATA port multipliers in external DAS enclosures not playing nice. FWIW, I haven’t had those issues (I use a hardware RAID array with an eSATA or USB connection).

My recollection is that someone who was familiar with the FreeBSD device drivers had made some particular controller work well with some specific port multiplier at one point in time.

The problem is that there are a lot of crappy port multipliers out there, and lots of marginal AHCI controllers that weren't well-tested with them, and I'm guessing minimal ongoing efforts to support bugs and quirks in the FreeBSD driver.

This is in some ways like the way we push people to use LSI HBA's crossflashed to a specific version. If you get the right stuff, it works.

The problem is that, unlike SAS port expanders, which are generally decent, many SATA port multipliers are made by Taiwan Back Alley Silicon Co. who might barely manage to get the stuff working under Windows. So I'm not horribly shocked that @Constantin is having reasonable luck storing stuff on Windows.

 

[Aidan]

Thanks folks. I reckon the Snapshots and Replication path is the way I'll go. The second box is built as such, is there a benefit in creating the same pools, shares, accounts etc. on it as are on the main server? Should I just create a pool for the backups as a destination for replication? What are people's experiences in this? Thanks again.