ZFS Snapshots, Shadow copies, performance and fragmentation?

[diskdiddler]

Hi,

Embarrassingly despite using FreeNAS for over 2 years now, I only use snapshots on my jails, not my main data set.
My main reason was, I thought I had to restore an entire snapshot to recover files, but I've recently discovered some reddit and forum posts here, indicating we could browse the snapshots in explorer somehow?

1) Is this still possible? I'd prefer to actually browse a 'second copy' of the filesystem and copy files out of the snapshot (if I needed to grab data) rather than doing a snapshot recovery, or individual file "previous versions" stuff.

2) As for the extra disk space used by a snapshot, I assume that would only be the amount of data changed, per period, right? So if I manipulate 100GB of data every 24 hours, my average snapshot would be around 100GB, right?
(My server isn't really low on space, but I'd hate to accidentally fill it)

3)Finally, how does this work for file fragmentation and performance in general? I don't want to push my little server too much.

 

[toadman]

Embarrassingly despite using FreeNAS for over 2 years now, I only use snapshots on my jails, not my main data set. My main reason was, I thought I had to restore an entire snapshot to recover files, but I've recently discovered some reddit and forum posts here, indicating we could browse the snapshots in explorer somehow?

1) Is this still possible? I'd prefer to actually browse a 'second copy' of the filesystem and copy files out of the snapshot (if I needed to grab data) rather than doing a snapshot recovery, or individual file "previous versions" stuff.

Yes, you use "Previous Versions" in Explorer. Right click on the folder in question > Properties > Previous Versions and select the one you want. Open it and start browsing.

2) As for the extra disk space used by a snapshot, I assume that would only be the amount of data changed, per period, right? So if I manipulate 100GB of data every 24 hours, my average snapshot would be around 100GB, right?
(My server isn't really low on space, but I'd hate to accidentally fill it)

Correct, a snapshot only captures the changes from the previous snapshot.

3)Finally, how does this work for file fragmentation and performance in general? I don't want to push my little server too much.

Good question. I'm not sure. Probably depends on your workload and it's characteristics. But I can't see how keeping extra data around would help performance. I suppose it depends on what is does to your utilization percentage of the pool. If you still have plenty of free space you will probably not notice the performance difference. (My guess anyway.)

 

[diskdiddler]

Yes, you use "Previous Versions" in Explorer. Right click on the folder in question > Properties > Previous Versions and select the one you want. Open it and start browsing.

Is there a way to browse in a more literal sense, literally see a second share, or a folder at the base of the share called .snapshot or something? So I can just manually go inside it all.
What if I accidentally, entirely muck up a folder structure / naming convention and I'm in the middle of moving files and I've done all this 'stuff' and I just want to recover files?

Good question. I'm not sure. Probably depends on your workload and it's characteristics. But I can't see how keeping extra data around would help performance. I suppose it depends on what is does to your utilization percentage of the pool. If you still have plenty of free space you will probably not notice the performance difference. (My guess anyway.)

My pool, to admit, is very full but it's still got over 800GB free, so I was thinking of keeping only 1 or 2 snapshots, for 6 hours and for I dunno 24 hours and that's it - very conservative but a possible bacon saver.

Also, how do I DO all this in Windows 10 and FreeNAS 11? I need to enable it (obviously the snapshots need to be turned on, for a start?)

 

[toadman]

I am not sure how you would "see a second" share. Maybe someone else can chime in. But you can manually go in in the way I explained. In short each "previous version" is a read only copy. i.e. it's the snapshot view. So file recovery is easy. You are really just "opening" the folder in explorer like you would any other. Then you can copy out what you need.

Start with the Freenas docs (http://doc.freenas.org/11/storage.html#periodic-snapshot-tasks), they explain how to setup the snapshots pretty well. It can all be done with the GUI, as can setting up the SMB shares in the first place if not already enabled.

 

[diskdiddler]

I like the idea of being able to navigate to /data/folder1/pictures/woodworking/ and right click on desk.jpg and restore a previous version because I accidentally saved over it - that's great (preferably recover to a new name?)

However, I'd also like to be able to go to

/.zfs/snapshot/data/folder1/pictures/ and just browser the woodworking folder?
Heck, although I don't have the space for it

/.zfs/snapshot/2018-02-07-12:00/data/folder1/pictures/woodworking
or
/.zfs/snapshot/2018-02-06-12:00/data/folder1/pictures/woodworking

Does that make sense?

 

[toadman]

Yes, makes sense. But isn't what you describe above what "Previous Versions" actually does? You open up the previous version folder with the date you want and it opens as a folder in Explorer. Then you browse it just like you would any other folder. Or am I still misunderstanding?

(And I think previous versions only works on folders actually, not files. So if you want to view a previous version of a file you would open the previous version of the folder and then look at the file present in that folder.)

 

[diskdiddler]

I think I need to see some screenshots of how it looks.

 

[toadman]

So I have an SMB share from freenas called "userdata". If I am in explorer and I right click and select Properties, then select the Previous Versions tab I get the first picture userdata_properties.jpg. It lists all the versions, each of which corresponds to a snapshot in Freenas. If I then select one of those and click "Open", the selected version opens in a folder view in Explorer. See userdata_explorer.jpg. I am then free to browse the contents of the snapshot.

 

[diskdiddler]

So I have an SMB share from freenas called "userdata". If I am in explorer and I right click and select Properties, then select the Previous Versions tab I get the first picture userdata_properties.jpg. It lists all the versions, each of which corresponds to a snapshot in Freenas. If I then select one of those and click "Open", the selected version opens in a folder view in Explorer. See userdata_explorer.jpg. I am then free to browse the contents of the snapshot.

Hi Toadmann,

Ok so, hopefully this helps other people.
Your pics are showing what I'm after but I didn't KNOW that, it's difficult to explain why.

I think my real problem is, I needed to access this old data, through a tab, in a properties window, in Explorer.
Using other snapshot based systems long ago, I distinctly recall a different share with a different timestamp, or maybe a directory at the root which says like
\\server\data
\\server\data-2018-02-14
\\server\data-2018-02-13
\\server\data-2018-02-12
\\server\data-2018-02-11

Having to access this data, via a tab in a properties window is clunky but ultimately, I end up with what I want, an NAVIGABLE explorer window, of a full snapshot. (At least, I think that's what it is)



I've had FreeNAS for 3 years and while I knew it could do something similar to this, I was reluctant to enable it for performance reasons / space reasons, but I've got a very conservative snapshot schedule setup (so I better notice my mistakes as I do them!)



Final question, for some experienced FreeBSD / FreeNAS developers or users.
What about fragmentation? How does this work with snapshotting? Infact, how does it work with FreeNAS? is my NAS slowly over the years just a total mess or is it somehow NOT fragmenting files? Surely snapshotting encourages more likely file messes?

Thanks so much to all

 

[diskdiddler]

Here's a screenshot, explaining how it works - it shows a diff volume label for the disk showing snapshot time, it shows me as being on \\server\share\SHARENAMEAGAIN?
It has a different amount of files in the root directory (I had deleted two files, since the snapshot was taken)


So I really hope this helps someone else who was confused by this.

 

[toadman]

Using other snapshot based systems long ago, I distinctly recall a different share with a different timestamp, or maybe a directory at the root which says like
\\server\data
\\server\data-2018-02-14
\\server\data-2018-02-13
\\server\data-2018-02-12
\\server\data-2018-02-11

FYI - on the freenas server (not via Windows) the data is available similar to what you suggest. Let's say my userdata dataset I mentioned above is located at /mnt/tank/userdata. Then I can find every snapshot at (for example):

/mnt/tank/userdata/.zfs/snapshot
/mnt/tank/userdata/.zfs/snapshot/auto-20180213.2258-1y
/mnt/tank/userdata/.zfs/snapshot/auto-20180213.2248-1y
...

Note the .zfs directory does *not* show up with an ls -al /mnt/tank/userdata command. But if you cd /mnt/tank/userdata/.zfs into it you will find the /snapshot directory *is* visible at that point.

And obviously what you are seeing in Windows within the explorer properties tab is these snapshot directories listed.

 

[garm]

Your worries about space usage is maybe a bit exaggerated. You say you “manipulate” 100 GB per 24 hours. Do you mean that you modify 100 GB worth of files or do you add 100 GB of files? There is an important difference between them.

If you have a text file containing “my cat is evil” and change the file after a snapshot to “my cat is cute”, you will not have two text files. You still only have the one file, but you are able to put it together in two different ways. The space needed by ZFS to keep both is only taken up by “cute” (plus some overhead).

So do you really produce 100 GB of modifications to your files every 24 hours?

If you are adding 100 GB of new files between snapshots, the resulting space gain would only be the files being removed by that action. Again, do you really remove 100 GB worth of data each 24 hours?

 

[diskdiddler]

Your worries about space usage is maybe a bit exaggerated. You say you “manipulate” 100 GB per 24 hours. Do you mean that you modify 100 GB worth of files or do you add 100 GB of files? There is an important difference between them.

If you have a text file containing “my cat is evil” and change the file after a snapshot to “my cat is cute”, you will not have two text files. You still only have the one file, but you are able to put it together in two different ways. The space needed by ZFS to keep both is only taken up by “cute” (plus some overhead).

So do you really produce 100 GB of modifications to your files every 24 hours?

If you are adding 100 GB of new files between snapshots, the resulting space gain would only be the files being removed by that action. Again, do you really remove 100 GB worth of data each 24 hours?

I totally don't make huge changes every 24 hours, for days, weeks, months on end.
HOWEVER, now and then I spend a day or two and re-sort and clean up hundreds of GB of mess, all very suddenly.
Whewn do I could move a LOT of data, hence me worrying.

I'm glad it's only diff data and not the whole lot.

Also, don't tell anyone else, cause they'll yell but I've far exceeded the 80% threshold for over ... 12 months now. Infact my highest usage has been 94% with only 600GB free (not much for my 6 drives)

 

[millst]

Like most filesystems, moving files on a dataset is a metadata change. You're just changing the directory pointer for the file.

-tm