How to make use of external drives with TrueNAS SCALE?

[derlinus]

Hi Arwen, yupp from a business perspective it may make less sense or maybe it does... No idea. I can just bring to the table that Qnap / which i don't like to use for multiple reasons has such thing as a button on the front of a device. Your backup HowTo is exactly what absolutely works for me too, even though I've stepped through this on the UI. Having amazingly sizable HDDs these days, i could well imagine that for a typical soho and home installation this would be a thing. Benefiting from iX graceful licensing there is absolutely nothing one can complain about.
Being in IT for too long, i don't like cloud backups that are being propagated by SiliconValey business models. I prefer to have a disk that i can put into my friends house or similar. For a business, having an offline and offsite backup can actually provide value with today's ransomware waves.
As mentioned with compression and todays drive sizes, this totally may be an option. Problem is: if a backup is not automated AKA too complicated cause difficult to remember what's required it's not necessarily great in practice. At least that's my opinion due to my experience in reality.
So in an attempt to raise awareness for the need, i thought i'll throw it out there, hoping that iX picks up on it and likes the idea. One can hope

 

[Arwen]

The difference between iXsystems & QNAP, is that QNAP only sells pre-packaged NASes with software. They don't allow installing their software on any old computer, (as far as I know). TrueNAS Core & SCALE are both Open Source and free for use by anyone, even on non-iXsystems hardware. We sort of act as their testers, Alpha or Beta, depending on the user and their need for specific features or bug fixes.

While it might be nice to have a Core jail tool, and a SCALE app, that does automated backups, their are a TON of gotchas. People might say if the drive is plugged in, import the pool, do the backup and export the pool.

Except that as the tool resets to waiting, it will re-detect the pool, re-import it, re-run the backup, export the pool, ad nauseam. So, you might say, wait a hour. Except what if someone does not remove the backup set within that hour? So then you say wait a day. Except what if they need to run a separate set of backups before end of quarter or a holiday?

Their are so many issues that require fault tolerance. Like pool scrubs. Should be done perhaps once a month. And my backup script automatically does such each time, since I don't run backups more often. However, in an automated backup, it should keep track of when the last backup set pool scrub was run and run one if needed.

Anyway, I can write a LOT of the gotchas, having experienced tons personally and on the job. But, I don't have time to write a tool for other people that I won't actually use myself. (Unless it's a perfect tool...)

 

[opensourceFTW]

Are replication tasks completed fast, like 5 min for 3.2TB? I will be replacing my TrueNAS server so I want to backup all of my data files to an external HDD. After the replication was completed, I took the external HDD to my comp to see what was on it. My comp cannot mount it, incorrect file system (ZFS).

Did my replication complete successfully?

 

[NugentS]

Did it say it was complete?
You are replicating one pool to another - the destination pool is also ZFS and in theory identical to the source pool. To test the destination pool you would need to mount it in something that supports ZFS.

 

[opensourceFTW]

Yes it said it was finished. I see that I do not understand replication. Under snapshots, it shows the replication, all 0.10 bytes of it.

How can I backup my files?

 

[NugentS]

How much space is used on the disk / dataset

 

[opensourceFTW]

How much space is used on the disk / dataset

2.4TB

I really need to update my home network. I decide to start copying the files from TrueNAS, through my PC, and into my external HDD. 1.8MB/s. 154ish hours for one folder. I am scared to copy my plex folder lol. Yikes.

 

[NugentS]

Assuming that's on the target - then you have backed up 2.4TB. The reason the last snapshot is small is because its taken just before the replication so there are unlikley to be any changes after it. When you next replicate there will another very small snapshot and the replication process will copy the files (actually blocks) over that have changed since the previous snapshot

Your target dataset will / should be ReadOnly - but you should be able to see the files in there via the shell

 

[derlinus]

Hi guys,
is therer a way to vote for feature requests to iX Systems ?

I would like to find enough people in the community to support a feature request to iX to become relevant enough to raise priority.
Basically it's about the same topic described in this thread, but also here.

Having a simply way to backup your NAS config, boot volume and selected content off box, off site and offline is something that should be accessible to everyone. Only if iX is doing it, i would trust this to work in the long run.

Would be great if there was a way to collect votes to make importance tengible to iX.

-Linus

 

[Arwen]

Hi guys,
is therer a way to vote for feature requests to iX Systems ?

I would like to find enough people in the community to support a feature request to iX to become relevant enough to raise priority.
Basically it's about the same topic described in this thread, but also here.

Having a simply way to backup your NAS config, boot volume and selected content off box, off site and offline is something that should be accessible to everyone. Only if iX is doing it, i would trust this to work in the long run.

Would be great if there was a way to collect votes to make importance tengible to iX.

-Linus

You have to add a Jira bug / feature request, then post the number, (preferably with link too), in all the relevant Forum posts. I'd vote for such a feature request, if the App / Jail program had suitable features for me;

• How often to run scrub
• What to backup, based on pool name
• Separate pool creation, to allow both multiple disks for larger backups. And specific features, like compression.
• Potentially support encryption on import. Automatic if using keys, prompt if using passphrase.
• Perhaps short SMART tests before starting backup. May not be possible on some USB.
• Perhaps check disk temperature during backup, and pausing if drive(s) get too hot. May not be possible on USB.

 

[opensourceFTW]

I finally got TrueNAS core up and running so now, is it possible to plug in my 5TB external drive that has all of my data from my last TrueNAS server so that I can move my data back on my new TrueNAS server?

I checked under import disks, it shows my 5TB drive. Under pools it said the only option was stripe. I stopped there as I did not want to lose my data.

 

[NugentS]

You can import a disk Storage/Import Disk assuming its formatted in an acceptable manner

 

[sretalla]

Just to be clear... you don't use the import disk process for a ZFS pool (single disk or otherwise)... use Add Pool, import an existing pool.

 

[NugentS]

As @sretalla corrects me.
If the disk was a ZFS pool on its own - then import the pool via Add and then import
If however is an NTFS / ext2 disk then you use what I told you

 

[opensourceFTW]

Just to be clear... you don't use the import disk process for a ZFS pool (single disk or otherwise)... use Add Pool, import an existing pool.

The external drive is formatted with ext4.

I backed up my files on that drive through my main PC. There was no ZFS pool.

 

[sretalla]

The external drive is formatted with ext4.

I backed up my files on that drive through my main PC. There was no ZFS pool.

OK, so @NugentS had the right idea then.

The problem you'll have is that the import disk routine doesn't support ext3 or 4, only ext2.

You'll be best to connect that drive to a system with Linux on it and copy the contents over the network with rsync.

 

[opensourceFTW]

OK, so @NugentS had the right idea then.

The problem you'll have is that the import disk routine doesn't support ext3 or 4, only ext2.

You'll be best to connect that drive to a system with Linux on it and copy the contents over the network with rsync.

I havent been able to figure out permissions because I cannot get my main PC to mount a share folder under TrueNAS core so that will have to wait.

 

[danb35]

The problem you'll have is that the import disk routine doesn't support ext3 or 4, only ext2.

But isn't ext3/4 just ext2 with extra features? I'd understood there was a good degree of backward compatibility, which should mean an ext3/4 disk should be able to be imported--though I'll admit I haven't tested it.

 

[Arwen]

When EXT4 was created, it allowed a different storage allocation scheme, that breaks backward compatibility. I vaguely recall this being extent based, verses the standard cluster based. This was done because extent based, (if I remember correctly), was faster, with less breaks in a file's data blocks. (Aka holes for new cluster I-Nodes... again, IIRC.)

All that said, again in theory, EXT4 could still use the old scheme.


Love it, or hate it, Linux does it's own thing.
(Unless it has to be standards compliant.)

 

[sretalla]

But isn't ext3/4 just ext2 with extra features? I'd understood there was a good degree of backward compatibility, which should mean an ext3/4 disk should be able to be imported--though I'll admit I haven't tested it.

I have seen threads here where the OP can't get a drive to import due to it being ext4. it usually ends with the OP wanting to install ext4 compatibility... the usual recommendations not to mess with the appliance and then some kind of advice to do a network copy from a linux system.