Rob Shaver: Trying to discover if FreeNAS is what I need.

[Robert Shaver]

My name is Rob Shaver and I'm a retired electronics engineer who also likes to shoot and edit video. I have many hard disks from many past computers, but my working storage is two 4GB and one 5GB USB 3 external drives. I install all my apps on the 5GB boot drive but try not to put any important files on that drive. All my data goes onto the external hard drive. I also have three more external USB 3 drives for backup because I always buy them in pairs.

Because backing up is a manual operation, I don't back up as often as I should. I use Sync Toy on each drive once in awhile. I take the computer off the LAN to avoid getting a ransomware infestation while I have my backup disks connected. So this is why I'm looking for a better solution.

I have scanned through the FreeNAS docs and been reading the forums for a couple of days now. I have not found any sign of FreeNAS having a system for backing up. So that brings me to my questions (please tell me if I should post this in another forum):

1. Does FreeNAS have a backup solution? (Correct me if I'm wrong but snapshots will let me recover files but are not really backups, right? If ransomware strikes I would not be able to recover the files, right?)
2. Is FreeNAS Corral a likely solution? (I can't really find any documentation about what Corral is. All I find is warnings not to use it yet.)
3. Is there any document that tells what Corral is in a nutshell?
4. Is it possible to divide the storage installed in a single FreeNAS server into two parts, one for daily use and another for backup?
5. If so, can I make the backup set off-line after each backup?
6. If I can take disks off-line, can they be spun down while not in use?
7. If all of this is possible, can it all be automated? (I'm pretty sure if we can do the rest, automating it won't be that hard.)
8. Is there some other product I should be looking at instead?

FreeNAS sounds pretty useful. I'm hoping that I can use it to solve my issues.

Peace,

Rob:-]

 

[Scareh]

I'm presuming you mean TB and not GB in disk sizes :p

other then that:

1) FreeNAS doesn't have a backup solution in place. Not in the sense you mean.
You *can* do a rsync towards another box, and there are ways to backup to the cloud (crashplan is most used on here).

2) forget you ever heard of FreeNAS Corral, in short, it doesn't exist anymore and should not be used. Use instead the current version: 9.10.x (download from the main FreeNAS page).

3) you can make different pools where you can backup from one pool to another. But since they are in the same server don't really know if it's that much benefit.
I presume since ransomware spreads via samba shares you could potentially save your data from being ransommed if you make 2 pools and only have smb shares on 1 pool. While rsyncing from pool to pool.
But it would be wiser to just invest in a decent anti-virus/malware/.. program, or just refrain from downloading from dodgy sites/email attachments :p

4) there are numerous topics on the forum where the advantages and disadvantages on spinning down disks are discussed. The general consensus is that it isn't worth it (energy wise) and that it puts more strain on the disk leading to earlier deads.


Hope that summarises it a bit

 

[SweetAndLow]

Snapshots are exactly what you want for ransomware protection. If someone encryption you're files you just roll back to the previous snapshot when they are not encrypted. Would take 1min of time. At the same time you should also have a backup. This is where you need to figure out what you want. So you want to run a program that connects to some cloud service for backup, do you want so application that back up to another server or do you want to use the built in replication feature for backup?

Sent from my Nexus 5X using Tapatalk

 

[Robert Shaver]

Thanks for the quick response Scareh. Yeah, I meant TB. Duh.

I do have anti-virus/malware/ software running ... but so did some of the people who got infected with the ransomware. I've also heard and read that anti-virus/malware/ software enlarges your attack surface so I haven't decided whether or not to keep it.

I'm also aware of the dangers of promiscuous web surfing and do tread carefully. In addition, one of my email addresses has been getting phishing attempts for about a month in the tens of messages every day. So far I've not opened any, but who knows, I could make a mistake. Right now, after I back up, I'm 100% protected.

So your idea of having two pools is exactly what I was thinking of. Having one working pool and another backup pool of the same size. But the rsync docs say it copies over a network, but I want both pools in the same server. (I don't see any advantage in having two servers. If the server fails I can replace the hardware and bring it up. If it's down for a week it would not be a disaster.)

Spinning the disks down is a minor issue. I was just thinking that, if the disk isn't spinning, it can't get corrupted/encrypted by ransomware.

SweetAndLow, here's the docs I found about snapshots:

A periodic snapshot task allows you to schedule the creation of read-only versions of ZFS volumes and datasets at a given point in time. Snapshots can be created quickly and, if little data changes, new snapshots take up very little space. For example, a snapshot where no files have changed takes 0 MB of storage, but as you make changes to files, the snapshot size changes to reflect the size of the changes.

Snapshots provide a clever way of keeping a history of files, should you need to recover an older copy or even a deleted file. For this reason, many administrators take snapshots often (e.g. every 15 minutes), store them for a period of time (e.g. for a month), and store them on another system (e.g. using Replication Tasks). Such a strategy allows the administrator to roll the system back to a specific time or, if there is a catastrophic loss, an off-site snapshot can restore the system up to the last snapshot interval.

So this does sound good but does not tell me how it accomplishes this magic. If I make a snapshot of a pool and store it on another system (or pool?), when is the data copied to that other system? I'm guessing that when a file is changed after a snapshot then the changed data is stored in a new location and the unchanged sectors are added to the snapshot.

But I don't want to have to reverse engineer how FreeNAS works or read the code to find out how it works. I'd like to have documentation that tells me how this works. Is there somewhere I can read more about this is accomplished?

Thank you all for your help.

 

[m0nkey_]

The user guide for FreeNAS 9.10 can be found at http://doc.freenas.org/9.10/freenas.html

I was just thinking that, if the disk isn't spinning, it can't get corrupted/encrypted by ransomware.

Disks can corrupt for different reasons. It's generally understood that spinning down/up puts additional strain on the drives.

You can protect yourself from ransomware by keeping regular backups and having snapshots enabled. Setting a snapshot to run two or three times a day should be sufficient (obviously adjust this to suit your needs). Should your data then be encrypted by ransomware, it's a simple case of pushing a button and your data is restored.

 

[gpsguy]

No need to reverse engineer or read the code.

Since "snapshot" is a term used by other NAS and SAN vendors, you could do a Google search to get a general idea as to how they work.

In the simplest example with FreeNAS, both the base data and snapshots (changes) are stored on the same server. You can easily restore a file/files from any snapshot.

If you want additional redundancy you could setup a second FreeNAS server and replicate the snapshots (from server #1) to it.

 

[wblock]

So this does sound good but does not tell me how it accomplishes this magic. If I make a snapshot of a pool and store it on another system (or pool?), when is the data copied to that other system? I'm guessing that when a file is changed after a snapshot then the changed data is stored in a new location and the unchanged sectors are added to the snapshot.

But I don't want to have to reverse engineer how FreeNAS works or read the code to find out how it works. I'd like to have documentation that tells me how this works. Is there somewhere I can read more about this is accomplished?

A periodic snapshot is set up. This creates a snapshot of the data on the source machine periodically. Then a replication task is added. Every time a new snapshot is created, the snapshot is replicated to the target system. There are examples here: http://doc.freenas.org/9.10/storage.html#replication-tasks

 

[danb35]

Is there any document that tells what Corral is in a nutshell?

Corral was to be the next major release of FreeNAS after the FreeNAS 9 series. For a variety of reasons, it crashed and burned spectacularly, and it was very quickly discontinued. The next release will be FreeNAS 11, which is expected to be within the next few weeks.

 

[Robert Shaver]

Thanks for all the help folks.

gpsguy, good idea. I tried various Google searches. I finally settled on "difference between snapshot and backup" and found the following.

A snapshot is a point-in-time copy of data created from a set of markers pointing to stored data and is effectively a backup. Snapshots provide a variety of approaches that can supplement backup and provide rapidly accessible copies to which is it possible to roll back.

So what are the key snapshot variants? They include:

• Copy-on-write snapshot – Most snapshot implementations use a technique called copy-on-write, which makes an initial snapshot then further updates as data is changed. Restoration to a specific point in time is possible as long as all iterations of the data have been kept. For that reason, snapshots can protect against data corruption, unlike replication.
• Clone/split-mirror snapshot – Another common snapshot variant is the split-mirror, where reference pointers are made to the entire contents of a mirrored set of drives, file system or LUN every time a snapshot is made. Clones take longer to create than copy-on-write snapshots because all data is physically copied when the clone is created. There is also the risk of some impact to production performance when the clone is created because the copy process has to access primary data at the same time as the host.
• Continuous data protection (CDP) – CDP is a method of snapshotting that tracks and store all updates to data as they occur. Theoretically, this means CDP solutions can roll back to any point in time, down to the smallest granularity of update. But there is a price to pay with CDP in terms of the cost of storage needed to keep every changed block copy and the performance impact of storing the data. As a result, some vendors implement what they call near-CDP, taking snapshots of changed data at set times and consolidating changes over a longer time period. This means heavily updated data doesn’t overwhelm the capacity of the CDP system. In virtual environments, APIs such as vSphere’s VADP enable CDP solutions to be implemented by third-party software vendors.

Can anyone tell me if freeNAS uses one of these methods for the snapshot feature?
When I searched specifically for ZFS Snapshot I found this:

Overview of ZFS Snapshots
A snapshot is a read-only copy of a file system or volume. Snapshots can be created almost instantly, and they initially consume no additional disk space within the pool. However, as data within the active dataset changes, the snapshot consumes disk space by continuing to reference the old data, thus preventing the disk space from being freed.

ZFS snapshots include the following features:

• The persist across system reboots.
• The theoretical maximum number of snapshots is 264.
• Snapshots use no separate backing store. Snapshots consume disk space directly from the same storage pool as the file system or volume from which they were created.
• Recursive snapshots are created quickly as one atomic operation. The snapshots are created together (all at once) or not created at all. The benefit of atomic snapshot operations is that the snapshot data is always taken at one consistent time, even across descendent file systems.

Snapshots of volumes cannot be accessed directly, but they can be cloned, backed up, rolled back to, and so on.

The last line was interesting. My guess is that, if I was hit by a ransomware attempt to encrypt all files then the snapshot would keep the old data from being changed and the new encrypted data would fill the disks. If the storage where above 50% when the attack started, the disks would fill up entirely and FreeNAS would have to stop with some kind of error.

Thanks again for your help.

 

[danb35]

Can anyone tell me if freeNAS uses one of these methods for the snapshot feature?

ZFS is a copy-on-write filesystem, and thus uses copy-on-write snapshots.

 

[Jailer]

My guess is that, if I was hit by a ransomware attempt to encrypt all files then the snapshot would keep the old data from being changed and the new encrypted data would fill the disks. If the storage where above 50% when the attack started, the disks would fill up entirely and FreeNAS would have to stop with some kind of error.

That's not how snapshots work. They revert the data back to to exactly what it was when the snapshot was taken, it doesn't copy any new data.

ETA: I had a hard time understanding the concept of snapshots when I first started out with FreeNAS. Someone once posted that an easy way to understand snapshots is to consider each snapshot a layer of glass. The first layer will be a duplicate of your data. Now lets say you add a document to your share and a new snapshot (layer) is created. This new snapshot will only contain the document you added and not all of the original data. This process continues as you accumulate snapshots. When you hold the glass upright and look at them together all your data is there but each layer only contains the changes at that point in time.

When you want to restore a snapshot it restores the data of the accumulated snapshots as it was in that point in time and discards anything newer than that date. So if you are infected with ransomware on Monday you can roll back to Sundays snapshot and it's all gone because it didn't exist on your dataset on Sunday.

I hope that helps and wasn't a blob of incoherent blather.

 

[Ericloewe]

That's not how snapshots work. They revert the data back to to exactly what it was when the snapshot was taken, it doesn't copy any new data.

Just to clarify: the pool will grow by 100% if all of it is suddenly encrypted between snapshots.

 

[Robert Trevellyan]

A snapshot is a read-only copy of a file system or volume.

This is why a snapshot protects against ransomware (emphasis adjusted). Ransomware could infect your computer at time A, then work on encrypting your data, including your periodically connected backup drives, regardless of whether your computer is connected to the internet, until you notice at time B. As long as you have a snapshot from before time A, you're all set.

 

[Robert Shaver]

So I've got about 12TB of data now on various hard drives, not counting backups. Let's say I build a freeNAS server with 20TB of storage, not counting backup. I make a snapshot at time A before infection. Then, at time B, I catch a ransomware virus which starts encrypting my NAS. The old data is not touched but the new encrypted data being created by the bug begins to fill up that free 8TB. Those 8TB are going to fill up before the bug can finish the whole job. What does freeNAS do when the entire free storage space becomes exhausted? Will I be able to do a roll-back after all storage is full? Has this been tested?

 

[SweetAndLow]

So I've got about 12TB of data now on various hard drives, not counting backups. Let's say I build a freeNAS server with 20TB of storage, not counting backup. I make a snapshot at time A before infection. Then, at time B, I catch a ransomware virus which starts encrypting my NAS. The old data is not touched but the new encrypted data being created by the bug begins to fill up that free 8TB. Those 8TB are going to fill up before the bug can finish the whole job. What does freeNAS do when the entire free storage space becomes exhausted? Will I be able to do a roll-back after all storage is full? Has this been tested?

Your system will not fill up until there is a snapshot taken of the encrypted data. At that point it will look like all the old data got deleted and the original snapshot will reference all the old data and take up space. At the same time the new snapshot will reference all the encrypted data thus doubling the used space. If this fills your pool the randsome wear will get a no space in disk error and probably stop. Filling a zfs pool is kind of tricky because you can't even delete stuff once it's filled. So you will have to truncate an encrypted file until you have a couple gigs of space. Then you can safely rollback your snapshot.

Sent from my Nexus 5X using Tapatalk

 

[Robert Trevellyan]

Keep in mind, ransomware could only encrypt connected shares, and you would presumably investigate a sudden dramatic loss of free space. It would take a non-trivial amount of time to encrypt 8TB over a network share.

EDIT: and FreeNAS will warn you at 80% full, even if you didn't notice before that.

 

[Jailer]

Or if you're that concerned about it set a quota on your dataset.

 

[danb35]

Or if you're that concerned about it set a quota on your dataset.

...or a separate dataset with a reservation.

 

[Stux]

The theoretical maximum number of snapshots is 264.

2^64

 

[Stux]

My name is Rob Shaver and I'm a retired electronics engineer who also likes to shoot and edit video. I have many hard disks from many past computers, but my working storage is two 4GB and one 5GB USB 3 external drives. I install all my apps on the 5GB boot drive but try not to put any important files on that drive. All my data goes onto the external hard drive. I also have three more external USB 3 drives for backup because I always buy them in pairs.

Because backing up is a manual operation, I don't back up as often as I should. I use Sync Toy on each drive once in awhile. I take the computer off the LAN to avoid getting a ransomware infestation while I have my backup disks connected. So this is why I'm looking for a better solution.

I have scanned through the FreeNAS docs and been reading the forums for a couple of days now. I have not found any sign of FreeNAS having a system for backing up. So that brings me to my questions (please tell me if I should post this in another forum):

1. Does FreeNAS have a backup solution? (Correct me if I'm wrong but snapshots will let me recover files but are not really backups, right? If ransomware strikes I would not be able to recover the files, right?)

It provides tools which will allow you to implement a variety of backup techniques/approaches

Snapshots, replications, rsync, etc

1. Is FreeNAS Corral a likely solution? (I can't really find any documentation about what Corral is. All I find is warnings not to use it yet.)
2. Is there any document that tells what Corral is in a nutshell?

Corral has been retconned. It never existed ;)

1. Is it possible to divide the storage installed in a single FreeNAS server into two parts, one for daily use and another for backup?

Yes.

Some people even have backup scripts which mount a USB external, then backup to the external. @Arwen

1. If so, can I make the backup set off-line after each backup?
2. If I can take disks off-line, can they be spun down while not in use?
3. If all of this is possible, can it all be automated? (I'm pretty sure if we can do the rest, automating it won't be that hard.)

See above

Is there some other product I should be looking at instead?

Don't think so.

FreeNAS sounds pretty useful. I'm hoping that I can use it to solve my issues.

Peace,

Rob:-]

FreeNAS is going to solve issues you don't even realize you have yet...