format and mount UFS drive for backup FreeNAS 9.3

[keithg]

The instructions I found on the net were outdated on how to do this and did not work correctly. I'm posting this here in case it helps someone.
I have a 4tb NAS 3 disk ZFS pool and wanted to make a snapshot backup on a hard drive so I had a backup of the data 'just in case'. I'll be periodically backing up to this, but at least I will have all my photos, etc from the BOT to now and the data set will be readable on Linux and I can rsync back to a new FreeNAS in case of catastrophe and my jails, etc are at least backed up somewhere.

I was able to hot plug a SATA disk to the machine. So, this is much faster than USB. It showed up as /dev/ada3. So I needed to format it and mount it as a UFS volume. I did this from the command line:

fdisk -BI /dev/ada3
bsdlabel -wB /dev/ada3s1
newfs -O2 -U /dev/ada3s1a
mount -t ufs /dev/ada3s1a /mnt/NAS_Backup/
rsync -av --progress /mnt/first_NAS/ /mnt/NAS_Backup/ &​

In the end I have a backup and was able to hot unplug it and put it in a safe place until next time. Would be nice if you could do this from the GUI.

Keith

 

[pirateghost]

That doesn't make sense to add something to the GUI that is being deprecated from FreeNAS (UFS support)

 

[keithg]

Ok, I was unaware of this. Are all filesystems other than zfs being deprecated? It is pretty nice to be able to plug a drive and read-only copy it to the NAS. and also it is so much faster than relying on the network for a rsync copy to do it locally. Is this capability going away, or just the ability to create and write UFS? I was able to read another drive previously this way. I do not know if it was ntfs or ext4, though.

Keith

 

[pirateghost]

As indicated in the documentation, the import feature is intended for read only access to drives other than zfs.

 

[Robert Trevellyan]

Ok, I was unaware of this. Are all filesystems other than zfs being deprecated? It is pretty nice to be able to plug a drive and read-only copy it to the NAS. and also it is so much faster than relying on the network for a rsync copy to do it locally. Is this capability going away, or just the ability to create and write UFS? I was able to read another drive previously this way. I do not know if it was ntfs or ext4, though.

The part about connecting a drive via SATA for backup is sound, it's only the part about using UFS or anything else that's legacy. FreeNAS no longer supports anything but ZFS for pools and the boot device. Some other formats are supported for import-only.

You can easily backup to a hot-plugged drive by creating a pool on that drive and using zfs replication or rsync or whatever you like. If you set that pool up correctly (all features disabled is the simplest), you can even mount it on Linux using zfsonlinux.

 

[danb35]

There's no reason you couldn't just create a ZFS pool on the backup disk, then import and export it as necessary from the GUI. The copying (or replication) itself would still need to be done from the CLI, though.

 

[JoeVulture]

There's no reason you couldn't just create a ZFS pool on the backup disk, then import and export it as necessary from the GUI. The copying (or replication) itself would still need to be done from the CLI, though.

Can the ZFS pool be imported/exported from the CLI?

I have the process that keithg is using completely automated via a script running as a cron job. I format the drives using UFS with a label name so that I don't have to worry about drive mount points changing (e,g. ada5 could become ada6) and the script will mount/unmount the drives after every rsync. Every Monday morning, the drives (two 3TB drives, each hold parts of my ZFS volume) are formatted by the script, so I swap them out Sunday night.

Thanks,
Joe

 

[danb35]

Can the ZFS pool be imported/exported from the CLI?

Sure, with zpool import poolname/zpool export poolname.

 

[JoeVulture]

Sure, with zpool import poolname/zpool export poolname.

I admit that was a stupid question. :) Mainly I'm looking to avoid doing anything behind FreeNAS' back, and by using UFS I can do that.

If I follow the UFS logic (which I actually adapted from an old Linux script I wrote), I'd be importing/exporting the pool every day (although maybe with ZFS, I'd want to not do that in order to allow ZFS to work its magic). My main concern would be destroying/recreating the ZFS pool on a weekly basis (analogous to reformatting the drive in UFS) via CLI commands.

Something to look into, I guess.

 

[Robert Trevellyan]

My main concern would be destroying/recreating the ZFS pool on a weekly basis (analogous to reformatting the drive in UFS) via CLI commands.

Why would you destroy and recreate the pool? In your scenario, you would only be detaching and importing it. The purpose of ZFS replication is to make the destination match the source, which it can do very quickly and efficiently if the differences are small. I hesitate to compare it to rsync, but for the purpose of understanding how you might use it efficiently, there are similarities.

 

[JoeVulture]

Robert,

Please bear with me; I'm trying to come up with an equivalent method to what I'm currently doing using ZFS - it's a different way of thinking for me. The ideas of how to treat my data in the pool that is always in the server make sense to me, but how to make it removable is throwing me a for a loop.

My script uses rsync to make incremental backups based on a day of the week; I format the drives on Monday and then let rsync make deltas so that I can go back to a previous day if need be (e.g. I realize that I messed something up for Wednesdays backup, I can go back to Tuesday). Perhaps this idea is not as useful as it used to be since I have snapshots enabled on my pool in the server. (I developed the script years ago when I was using Linux with ext2.)

It sounds like I might want to do something like this:

• Create pools of one drive, all named the same thing on my backup drives
• At the beginning of the backup cycle (week), import the pool(s)
• Use ZFS replication (configured through the GUI) to push certain parts of my main volume to the backup drives
• At the end of the backup cycle, detach the pool and swap in the next set of drives

With this method, I suppose I lose the ability to version (by day of the week) of my files on the backup drives (maybe using snapshots would work?). Whatever method I use, I just don't want to fill the drives, hence why I format them every week so that I have a clean slate.

Thanks,
Joe

 

[Robert Trevellyan]

It sounds like I might want to do something like this:

• Create pools of one drive, all named the same thing on my backup drives
• At the beginning of the backup cycle (week), import the pool(s)
• Use ZFS replication (configured through the GUI) to push certain parts of my main volume to the backup drives
• At the end of the backup cycle, detach the pool and swap in the next set of drives

With this method, I suppose I lose the ability to version (by day of the week) of my files on the backup drives (maybe using snapshots would work?). Whatever method I use, I just don't want to fill the drives, hence why I format them every week so that I have a clean slate.

A combination of ZFS snapshots and replication can handle the whole process very smoothly, except for the part where you rotate the backup drives weekly. However, unless the goal of rotating the drives is to move a copy offsite, you may be willing to give that up. There are three key components to this:

1. Automated snapshots on the main pool at a suitable interval, with automatic expiry. In your case, it might be daily snapshots with weekly or bi-weekly expiry.
2. A second pool used as an automatic replication target. That pool should have some redundancy (hence you might be willing to give up rotation). It might be a mirror of two large drives, or a RAIDZ1 or RAIDZ2 pool of 3, 4 or more drives.
3. A replication task that incrementally replicates snapshots from the main pool to the second pool.

The idea is that the replication task periodically makes the contents of the second pool match the contents of a snapshot of the main pool, including unexpired snapshots, and deletes expired snapshots. The automatic expiry of snapshots ensures that storage utilization doesn't grow indefinitely on either pool.

If you're determined to rotate the backup disks, it's harder to automate, but no doubt it can be done.

 

[Robert Trevellyan]

I see from the other thread you posted in that you are rotating backups offsite, which makes automatic replication more tricky. However, you could combine daily snapshots with weekly manual replication. You would use appropriate command-line flags on the replication command to ensure that expired snapshots were deleted from the backup.