USB Drive as offline backup

[harsh]

Please take five minutes to go back and read what I wrote instead of assuming my intent. I'll restate it again since you seen to have missed it.

Have you tried testing rsync from one pool to another within TrueNAS? Hopefully you can easily add a single drive to do so. That way you can determine the maximum rsync performance with no other variables such as usb, network, etc.

I didn't miss your point. I'm waiting for the current archive to complete before I start experimenting further.

If rsync is slow on Truenas, then Truenas is likely out the door.

 

[harsh]

A better method is to use size plus date & time stamp as I've shown above.

I tried this method initially and it didn't appear to be any faster. I suspect that since there aren't any files on the destination, there's nothing to compare to (at least yet).

 

[Arwen]

I tried this method initially and it didn't appear to be any faster. I suspect that since there aren't any files on the destination, there's nothing to compare to (at least yet).

Yes, if the destination is empty, then yes, it would not mater. RSync would have to copy all the files.

If that is your normal case, empty destination, then a different program might be better to perform the copy.

 

[harsh]

If that is your normal case, empty destination, then a different program might be better to perform the copy.

Suggestions? rsync is just soooo easy and tools like tar create unwieldy files that must be extracted/restored.

This isn't a normal case. This is a case where I want to start over with my Truenas installation to confirm that it is (or isn't) suitable for my server's hardware. If it turns out that Truenas doesn't get along with the server, I need to be able to copy the files to a different NAS setup and/or use them straight off the USB drive.

 

[winnielinnie]

You might get better performance if you rsync over the local network with your NTFS USB drive connected to a Windows PC.

 

[CJRoss]

I didn't miss your point. I'm waiting for the current archive to complete before I start experimenting further.

If rsync is slow on Truenas, then Truenas is likely out the door.

Who said anything about rsync on TrueNAS? I'm talking about rsync in general. If you're going to be this combative when people try to help you then I don't know why you bothered asking for help.

Yes, if the destination is empty, then yes, it would not mater. RSync would have to copy all the files.

If that is your normal case, empty destination, then a different program might be better to perform the copy.

Rsync will still read everything and create the checksums on the source AFAIK.

 

[harsh]

You might get better performance if you rsync over the local network with your NTFS USB drive connected to a Windows PC.

A Windows PC complicates things as privileges and ownerships start getting confused.

I do plan on testing with a drive connected to a Debian machine.

 

[harsh]

Who said anything about rsync on TrueNAS?

I did. I have a very specific and plainly stated application.

I'm talking about rsync in general.

In general, I've found rsync on other supported platforms to be about as fast as anything similar.

Rsync will still read everything and create the checksums on the source AFAIK.

It does but as I'm not seeing huge spikes in CPU usage, I'm not too concerned. Am I wrong in that?

If I seem snippy it is because my goal here is to solve a very specific problem. It is not to confuse/divert from my issue or be schooled about other technologies that may or may not improve my situation.

 

[Arwen]

...
Rsync will still read everything and create the checksums on the source AFAIK.

If you don't use the checksum command line option, then RSync does NOT read everything. It reads the directory for size of file and the day / time stamp. If those are the same, skip that file. This snipit is straight from the RSync manual page;

Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for
files that have changed in size or in last-modified time.
...

There is a caveat about "-c" checksums;

Note that rsync always verifies that each transferred file was correctly reconstructed on the re-
ceiving side by checking a whole-file checksum that is generated as the file is transferred, but
that automatic after-the-transfer verification has nothing to do with this option's before-the-
transfer "Does this file need to be updated?" check.

But, as the original poster stated, sometimes the destination is empty. In which case of course RSync reads everything, (but does not bother with the checksums because their is no destination file to compare it to).

 

[winnielinnie]

If I seem snippy it is because my goal here is to solve a very specific problem. It is not to confuse/divert from my issue or be schooled about other technologies that may or may not improve my situation.

I was trying to eliminate variables. One being that the bottleneck may be in fact the ntfs module in SCALE (i.e, Debian Linux), which might underperform for continuous writes, as opposed to native NTFS on a Windows PC.

A Windows PC complicates things as privileges and ownerships start getting confused.

If you just need the files/data saved outright, and don't care about permission and ownership preservation, why not try exFAT? (It receives firstclass status in the Linux kernel, thanks in part to Samsung.)

exFAT is also cross-platform compatible, and well supported across different OSes.


EDIT: There's also the natural slowdown from transferring many files of all sizes (especially smaller ones), as opposed to the same amount of total data represented by few, large files. This could explain the 28 M/Bs you're seeing.

 

[Say Heading]

Maybe use a completely different approach?
I'm in the same boat - sort of: Would like to backup to HFS or APFS. Currently would have to use my macOS client to create backups from the trueNAS shares via the network with e.g. Carbon Copy Cloner and devices attached to the client. This keeps me from diving deeper into the topic.

But what if one would just build a VM with compatible operating system on trueNAS which backs up to USB/SATA devices (directly attached to the trueNAS box), this way being able to use whatever filesystem is desired?

The longer I think about it (and haven't done much yet) the more elegant does it seem to be. Would this be an option?
(...and what OSes other than macOS that seems to be quite difficult to be used as VM would handle my needs would be worth a discussion in a different thread.)

 

[CJRoss]

If you don't use the checksum command line option, then RSync does NOT read everything. It reads the directory for size of file and the day / time stamp. If those are the same, skip that file. This snipit is straight from the RSync manual page;


There is a caveat about "-c" checksums;


But, as the original poster stated, sometimes the destination is empty. In which case of course RSync reads everything, (but does not bother with the checksums because their is no destination file to compare it to).

What I had been trying to say is that rsync still has to index the source and wouldn't immediately start a copy even if there isn't anything at the destination. Which can still take a bit if it's a lot of small files.

I was trying to eliminate variables. One being that the bottleneck may be in fact the ntfs module in SCALE (i.e, Debian Linux), which might underperform for continuous writes, as opposed to native NTFS on a Windows PC.



If you just need the files/data saved outright, and don't care about permission and ownership preservation, why not try exFAT? (It's receive firstclass status in the Linux kernel, thanks in part to Samsung.)

exFAT is also cross-platform compatible, and well supported across different OSes.


EDIT: There's also the natural slowdown from transferring many files of all sizes (especially smaller ones), as opposed to the same amount of total data represented by few, large files. This could explain the 28 M/Bs you're seeing.

That's why I had originally suggested a pool to pool rsync just to see if it's a limitation regarding the files and/or rsync and not anything to do with NTFS, etc.

Maybe use a completely different approach?
I'm in the same boat - sort of: Would like to backup to HFS or APFS. Currently would have to use my macOS client to create backups from the trueNAS shares via the network with e.g. Carbon Copy Cloner and devices attached to the client. This keeps me from diving deeper into the topic.

But what if one would just build a VM with compatible operating system on trueNAS which backs up to USB/SATA devices (directly attached to the trueNAS box), this way being able to use whatever filesystem is desired?

The longer I think about it (and haven't done much yet) the more elegant does it seem to be. Would this be an option?
(...and what OSes other than macOS that seems to be quite difficult to be used as VM would handle my needs would be worth a discussion in a different thread.)

That could still run into slowness depending on the type of data being backed up. As winnielinnie pointed out, many small files will be slow no matter what.

 

[Say Heading]

That could still run into slowness depending on the type of data being backed up. As winnielinnie pointed out, many small files will be slow no matter what.

Yes, the mistake in my logic is that the macOS VM would still have to access the shares via SMB/network, it would not be able to read (or write to) the pool directly like to a SATA/USB mounted device.

 

[CJRoss]

Yes, the mistake in my logic is that the macOS VM would still have to access the shares via SMB/network, it would not be able to read (or write to) the pool directly like to a SATA/USB mounted device.

I believe that's correct. I haven't done any VM hosting on Core and I've just started working with Scale.

 

[harsh]

Have you tried testing rsync from one pool to another within TrueNAS? Hopefully you can easily add a single drive to do so.

I tested this on a subset of the files going from an HDD array to an SSD array. The write speed was around 15MB/second (ten times that of USB 3.0).

The problem appears to be with the USB connection in TrueNAS.

 

[harsh]

If you just need the files/data saved outright, and don't care about permission and ownership preservation, why not try exFAT?

Why would you suggest that when I said that ownership and permissions were a concern?

 

[winnielinnie]

Why would you suggest that when I said that ownership and permissions were a concern?

Then why are you using NTFS on Linux? The Unix permissions and ownerships are only emulated.

If you want to preserve everything, you’ll need to use ZFS or another *nix filesystem.

 

[CJRoss]

I tested this on a subset of the files going from an HDD array to an SSD array. The write speed was around 15MB/second (ten times that of USB 3.0).

The problem appears to be with the USB connection in TrueNAS.

I don't think that's a valid test. You're going to one or more SSDs, which will of course perform better than a single HDD regardless of whether it's connected via USB or not. That's why I mentioned adding a pool comprised of a single HDD as a test. Additionally, if you're using a standard USB drive and not an enclosure that you put together, you're running into the case where the drive is probably lower binned and may even be SMR.

 

[harsh]

Then why are you using NTFS on Linux?

Because the top criterion for doing an archival backup is portability and ZFS isn't all that portable.

I'll likely be moving to some platform other than TrueNAS SCALE once this is done as TrueNAS SCALE seems to be headed in a more captive direction.

 

[Ericloewe]

Because the top criterion for doing an archival backup is portability and ZFS isn't all that portable.

Nonsense, it's the most portable filesystem capable of storing files larger than 2 GB (technically FAT32 is more widespread, but that's only still relevant because of UEFI).