USB Drive as offline backup

[gwaitsi]

Hey guys,
I know this has been asked many times under freenas/truenass but I am wondering if Scale can be used to make a difference.

I have a small home server with 2 drives one for media and one for data. I have a 3rd drive in the box and i have snapshots and replication setup. I replicate the datasets from the data/media drive to the backup drive and all seems to work good.

The problem is, i would feel much more comfortable if i can have a backup drive that can be rotated into a fireproof safe.
i.e. one a week/month to minimize the potential for total loss.

This seemed too difficult to achieve with Core, so I migrated to Scale sometime ago.
Can I have this desired outcome with Scale now?

 

[Arwen]

Why do you think it was difficult to achieve with Core?

How to: Backup to local disks

How to: Backups to local disks Overview: This subject comes up often enough that I thought I would start a post of how I do it. Specifically I back up my TrueNAS to a hot-inserted, locally attached disk, then when done, hot-remove and take...

www.truenas.com

When I started backing up my FreeNAS, (before the name change to TrueNAS), I was less familiar with ZFS Send & Receive, so I used my old, reliable RSync. By using RSync, I got the benefit of exact backup of the NAS without any of the snapshots taking up space. I then was able to use a ZFS Snapshot on the USB / eSATA backup disk to record that day's backup as a R/O snapshot. If I started running low on backup space, I simply destroyed the oldest snapshot(s) to make enough space for my new backup.

After all these years, I see little benefit to using ZFS Send & Receive for my backups. I have no time constraint for when the backups need to be completed.

Using a USB attached drive for offline backups should work similar between Core & SCALE. A lot of people think SCALE is an upgrade to TrueNAS, but in someways the opposite is true. Core is likely more stable and probably a little faster than SCALE.

 

[gwaitsi]

Why do you think it was difficult to achieve with Core?

How to: Backup to local disks

How to: Backups to local disks Overview: This subject comes up often enough that I thought I would start a post of how I do it. Specifically I back up my TrueNAS to a hot-inserted, locally attached disk, then when done, hot-remove and take...

www.truenas.com

When I started backing up my FreeNAS, (before the name change to TrueNAS), I was less familiar with ZFS Send & Receive, so I used my old, reliable RSync. By using RSync, I got the benefit of exact backup of the NAS without any of the snapshots taking up space. I then was able to use a ZFS Snapshot on the USB / eSATA backup disk to record that day's backup as a R/O snapshot. If I started running low on backup space, I simply destroyed the oldest snapshot(s) to make enough space for my new backup.

After all these years, I see little benefit to using ZFS Send & Receive for my backups. I have no time constraint for when the backups need to be completed.

Using a USB attached drive for offline backups should work similar between Core & SCALE. A lot of people think SCALE is an upgrade to TrueNAS, but in someways the opposite is true. Core is likely more stable and probably a little faster than SCALE.

can you provide a step-by-step points please. I don't think the drives auto mount right? do you need some scripts for it? unmounting? these are not options in the gui right?

 

[Samuel Tai]

can you provide a step-by-step points please. I don't think the drives auto mount right? do you need some scripts for it? unmounting? these are not options in the gui right?

Did you look at the HOWTO link @Arwen provided?

Basically, you create a single-disk pool on the external USB drive. You can do this via the GUI. Then you create replication tasks to replicate your datasets from your data pool to the USB pool. Once the replications complete, you export your USB pool so you can disconnect it. All this is within the capabilities of the GUI, and is what I do weekly myself to backup my system.

 

[Arwen]

can you provide a step-by-step points please. I don't think the drives auto mount right? do you need some scripts for it? unmounting? these are not options in the gui right?

Mine is all command line because that is easier for me. But either GUI or command line can achieve the same result.

@Samuel Tai listed the rough steps for the GUI.

Perhaps we need a new HowTo guide for the GUI. Though I don't have any energy to create & test one, that I won't be using.

 

[Cabby]

Did you look at the HOWTO link @Arwen provided?

Basically, you create a single-disk pool on the external USB drive. You can do this via the GUI. Then you create replication tasks to replicate your datasets from your data pool to the USB pool. Once the replications complete, you export your USB pool so you can disconnect it. All this is within the capabilities of the GUI, and is what I do weekly myself to backup my system.

Hi, out of curiosity, how long does it usually take? I got a HP Gen8 Microserver with four 3 TB disks in a raidz1 with around 6.5 TB used, probably a bit less size than the system in your signature.

I ran a replication task today for the first time, to a external 8TB 3.0 usb drive, and it is running for 12h and is not finished yet. The i/o speeds look like what you expect from earlier generation USB3, though.

Will the next back take up the same amount of time, or does this then generates smaller snapshots based on the first replication

Thanks for your help, if you can find the time.
Cabby.

 

[winnielinnie]

and it is running for 12h and is not finished yet.

Back of a napkin calculations, expect it to take around 20 hours. More or less, depending on other factors.

Will the next back take up the same amount of time

The next backup will only transfer the difference between the "base" snapshot and the most recent snapshot. If that means around 3 GB of new/modified data, then only 3GB needs to be transferred.

 

[harsh]

I have a large USB 3.0 drive connected to a Dell T330 server (via a USB 3.0 port) and it is astonishingly slow. Doing a 6TB backup (via rsync from the command line) is taking well upwards of two weeks.

I'm wondering if I have something set up wrong or there is a bad driver.

 

[Arwen]

@harsh - It could be a SMR drive.

What is the model of the drive?


Further, just because a drive and host port use USB 3.0, does not mean they use the newer block transfer protocol UASP, (USB Attached SCSI Protocol). UASP, (sometime abbreviated to UAS), is somewhat new and allows the faster transfer of data as well as disk management. The OS, USB driver and device need to support UASP for that feature to work.

The old USB block transfer protocol was not really designed for disks, as no one in their right mind would attach a disk at USB 1.0 speeds, (1.5Mbits/ps or 12 Mbit/ps). But, that was what was available when the first drives became available with USB interfaces. It basically persisted for far too many years, just like half duplex transfers.

 

[harsh]

It could be a SMR drive.

I'm pretty sure that isn't the issue. I can do large writes relatively rapidly on my desktop computers. In a PCMag review of the drive, Crystalmark 7 rates the drive at around 150MBps sustained sequential write speed which works out to about 12 hours to do the transfer (in a perfect world).

What is the model of the drive?

It is a Seagate Backup Plus Hub STEL8000100. It is perhaps a 6-7 year old design.

Further, just because a drive and host port use USB 3.0, does not mean they use the newer block transfer protocol UASP, (USB Attached SCSI Protocol). UASP, (sometime abbreviated to UAS), is somewhat new and allows the faster transfer of data as well as disk management. The OS, USB driver and device need to support UASP for that feature to work.

Given that both the server and the USB drive are of around a 2017 vintage, I'd be surprised if they aren't using sufficient hardware.

I haven't tried stock Debian on the server so I don't have a point of reference.

I will point out that every 30 minutes (like clockwork), one of the server's SAS drives (always /dev/sdg) reports a sector error.

 

[Arwen]

It does not mater in the least what other file systems can do speed wise with that drive. ZFS does COW, (Copy On Write), which uses free space to make new writes, and then freeing up any over-written data at the end. Thus, ZFS may bounce around the LBA, (Logical Block Addressing), instead of direct over write of data.

Actually that STEL8000100 does appear to be a SMR drive. From Google "Is STEL8000100 SMR?";

A: YES - the 8TB Seagate Expansion drive STEB8000100 and the 8TB Seagate Backup Plus drive STEL8000100 are both shingle / smr technology (per Seagate support).

While I personally own and use a Seagate 8TB SMR Archive drive, AND use it with ZFS for backup purposes, I don't leave it connected. It stays connected as long as needed for the backup procedure. Over time I have seen the drive become slower and slower. But, I am not backup time constrained.

As for your SAS drive reported sector error, it would appear the sector has not been spared out properly. Don't know why, but that would be for another post & subject.

 

[harsh]

It does not mater in the least what other file systems can do speed wise with that drive. ZFS does COW, (Copy On Write), which uses free space to make new writes, and then freeing up any over-written data at the end. Thus, ZFS may bounce around the LBA, (Logical Block Addressing), instead of direct over write of data.

It most certainly does matter as the write overhead of ZFS writes doesn't come into play on an NTFS formatted backup drive. I can't imagine why one would use ZFS on a USB drive.

Actually that STEL8000100 does appear to be a SMR drive. From Google "Is STEL8000100 SMR?";

That shouldn't significantly change its write performance from platform to platform as long as the filesystem remains the same.

While I personally own and use a Seagate 8TB SMR Archive drive, AND use it with ZFS for backup purposes, I don't leave it connected.

Other than using NTFS rather than ZFS, I use it in the same way. My intent was to do a full storage backup and start over; perhaps with Cobia. As it is, this gives me weeks to decide on that question.

As for your SAS drive reported sector error, it would appear the sector has not been spared out properly. Don't know why, but that would be for another post & subject.

Agreed. It does rise to the level of its own discussion.

 

[Ericloewe]

I can't imagine why one would use ZFS on a USB drive.

You just haven't been exposed to the right sort of ideas.

 

[Arwen]

It does not mater in the least what other file systems can do speed wise with that drive. ZFS does COW, (Copy On Write), which uses free space to make new writes, and then freeing up any over-written data at the end. Thus, ZFS may bounce around the LBA, (Logical Block Addressing), instead of direct over write of data.
...

It most certainly does matter as the write overhead of ZFS writes doesn't come into play on an NTFS formatted backup drive. I can't imagine why one would use ZFS on a USB drive.
...

What I meant was that ZFS is different enough, (and in some cases slower), from other file systems, that ZFS' write behavior on SMR disks is poor.

So I agree, the ZFS overhead for writes on a SMR disk is almost certainly slower than say NTFS.


As for using ZFS on a USB attached SATA disk, the biggest reason is simple - Data Verification. You get confidence that what you wrote is what you read. Part of my backup script is a ZFS scrub. At the end of the backup, I check the log file for errors and deal with any as appropriate.

Now of course my backup disks, (1 x 750GB CMR, 1 x 8TB SMR, 1 x 12TB CMR), are all SATA bare drives. I have drive swappable external enclosures with USB and or SATA interfaces, which I can use either interface as appropriate.

 

[harsh]

What I meant was that ZFS is different enough, (and in some cases slower), from other file systems, that ZFS' write behavior on SMR disks is poor.

Again, none of that matters in this thread as my backup drive is formatted NTFS. Shingled drives may be slower, but I'm seeing 1.53MB/second average performance on a drive that is reportedly capable of nearly 150MB/second sustained sequential writes.

Can we finally agree that ZFS isn't slowing down an NTFS-formatted USB drive?

 

[harsh]

As for using ZFS on a USB attached SATA disk, the biggest reason is simple - Data Verification.

I independently hash each file after the backup for verification. I trust that method more.

 

[Arwen]

Oh, NTFS formatted drives aren't supported by TrueNAS SCALE. At least as far as I remember. Just because it may work now, does not mean it's is either a supported feature. Or will be working in the future.

 

[CJRoss]

One thing to keep in mind is that if you're not diligent enough to rotate the drives it really won't matter if there's a fire. I know I'm not so I prefer an automated sync to a remote TrueNAS or a cloud backup.

 

[harsh]

Oh, NTFS formatted drives aren't supported by TrueNAS SCALE. At least as far as I remember. Just because it may work now, does not mean it's is either a supported feature. Or will be working in the future.

Remember that I'm talking about a backup drive and not a drive that is part of a storage pool. I am NOT writing to a storage pool here.

I don't recall whether I mounted the USB drive with NTFS or ntfs-3g. That may have something to do with it.

 

[dak180]

Remember that I'm talking about a backup drive and not a drive that is part of a storage pool. I am NOT writing to a storage pool here.

I don't recall whether I mounted the USB drive with NTFS or ntfs-3g. That may have something to do with it.

The only supported file system for TrueNAS is ZFS; plan for any others to be turned off at some point and locked out; if you want to use a backup drive formatted with another file system I would recommend setting up a workflow where you connect the drive to some other machine and transfer the data via a network protocol. The other option is a single device pool that is formatted with ZFS, this is what I use for my external backup.