deleting files does not free up space

[pitzav]

Hi I keep deleting files and I don't see any storage freeing up. I went to my snapshots and there are some snapshots that use very small amount of GB. And there is on very old that uses 211GB and referencing 982 GB.
The files I am deleting were added after this old snapshot, so I assume if I delete them they should be gone right?
My main problem is that I want to free up some storage because I receive a warning about using 92% of the pool.
I am also a little afraid of deleting this old snapshot file.

 

[SweetAndLow]

Why

Hi I keep deleting files and I don't see any storage freeing up. I went to my snapshots and there are some snapshots that use very small amount of GB. And there is on very old that uses 211GB and referencing 982 GB.
The files I am deleting were added after this old snapshot, so I assume if I delete them they should be gone right?
My main problem is that I want to free up some storage because I receive a warning about using 92% of the pool.
I am also a little afraid of deleting this old snapshot file.

Why are you afraid of deleting the snapshot? So you plan on rolling back to it? Your issue is most likely your snapshots.

 

[pitzav]

I am new and whatever has the word delete scares me. :) That big snapshot is from the day I set up the NAS and that's what seems strange.

So if I delete it will I free up space equal to the used space or equal to the referencing space?

 

[pitzav]

I deleted the very big snapshot and I have about 220gb more now. Now I am left with some very small snapshots. All are about 14gb used. The thing is that I still delete files now and I don't see space freeing up.

 

[pitzav]

Does it have something like a cron job to delete after an X amount of time maybe?

 

[Arwen]

Probably.

What you seem to want, is rolling snapshots so that you can recover accidentally deleted files. I have my desktop, (which also uses ZFS), perform 24, hourly rotating snapshots. So if I make a mistake, I can recover the file from an earlier snapshot. Plus, I take daily snapshots, kept for 7 days, in case I want something further back.

The number of snapshots, how often to take then, and how long to keep them, is more or less related to your need for recovering old files or rolling back an entire dataset / zVol. Plus, the amount of space you can dedicate to this function.

In my backup disk's case, I keep the snapshots until I reach about ~80% used. Then delete the oldest until I am below 80% used. This can give me additional backups of backups, though on the same medium. But, no need to leave unused space.

Some people run without any snapshots. Or use backup devices. All your choice.

 

[pitzav]

Probably.

What you seem to want, is rolling snapshots so that you can recover accidentally deleted files. I have my desktop, (which also uses ZFS), perform 24, hourly rotating snapshots. So if I make a mistake, I can recover the file from an earlier snapshot. Plus, I take daily snapshots, kept for 7 days, in case I want something further back.

The number of snapshots, how often to take then, and how long to keep them, is more or less related to your need for recovering old files or rolling back an entire dataset / zVol. Plus, the amount of space you can dedicate to this function.

In my backup disk's case, I keep the snapshots until I reach about ~80% used. Then delete the oldest until I am below 80% used. This can give me additional backups of backups, though on the same medium. But, no need to leave unused space.

Some people run without any snapshots. Or use backup devices. All your choice.

Thank you for the tips. :) But I am not sure when are the files actually get deleted so I can free up space. I have a periodic every 12 hours task and keep them for 1 week. Does that mean if I delete something today I will see that free space in a week from now?

 

[Arwen]

Okay, simple snapshot example.

You create files.
Make snapshot
Delete files.
No space freed up

Snapshots are an exact "copy", (if you like), of the ZFS Dataset at the time of the snapshot. Until the snapshot that contains the files is removed, you can always get the file back. BUT, it takes up space.

Keeping snapshots too long, can take up too much space because you may have added more files. And the files you have deleted have not freed up space, (because their is a snapshot hold on them.) That's up to the admin of ZFS.

To find out if a file is in your snapshot, (AND taking up space), mount the ZFS snapshot, (ZFS snapshots are READ / ONLY). Then see if the file exists. If so, easing the file in the parent Dataset won't free up any space. If you absolutely need that space back, your only recourse is to delete the snapshot.


Another way to look at it, (if you are familiar with hard links), is that a snapshot creates a R/O hard link to your files. Erasing the so called "parent" hard link does not free up space on disk because the other hard link still points to the file and all it's data.


Now for your question about your periodic snapshots. Yes, you have to wait up to a week before the space is freed up.

It all depends on WHEN you added that file. So, if you have added a file on Wednesday, and delete it today, (Saturday), you don't have to wait a week to see the free space. Only until Wednesday. (The exact time on Wednesday depends on when you added the file and when the snapshot was taken that INCLUDES that file.)

 

[pitzav]

Okay, simple snapshot example.

You create files.
Make snapshot
Delete files.
No space freed up

Snapshots are an exact "copy", (if you like), of the ZFS Dataset at the time of the snapshot. Until the snapshot that contains the files is removed, you can always get the file back. BUT, it takes up space.

Keeping snapshots too long, can take up too much space because you may have added more files. And the files you have deleted have not freed up space, (because their is a snapshot hold on them.) That's up to the admin of ZFS.

To find out if a file is in your snapshot, (AND taking up space), mount the ZFS snapshot, (ZFS snapshots are READ / ONLY). Then see if the file exists. If so, easing the file in the parent Dataset won't free up any space. If you absolutely need that space back, your only recourse is to delete the snapshot.


Another way to look at it, (if you are familiar with hard links), is that a snapshot creates a R/O hard link to your files. Erasing the so called "parent" hard link does not free up space on disk because the other hard link still points to the file and all it's data.


Now for your question about your periodic snapshots. Yes, you have to wait up to a week before the space is freed up.

It all depends on WHEN you added that file. So, if you have added a file on Wednesday, and delete it today, (Saturday), you don't have to wait a week to see the free space. Only until Wednesday. (The exact time on Wednesday depends on when you added the file and when the snapshot was taken that INCLUDES that file.)

Thank you so informative! It actually makes sense in my mind now. The confusing part was the used space I was seeing. If I added them up it was only a few GB and not the actual space.

 

[Arwen]

@pitzav, great.

I have a periodic every 12 hours task and keep them for 1 week. Does that mean if I delete something today I will see that free space in a week from now?

It all depends on WHEN you added that file. So, if you have added a file on Wednesday, and delete it today, (Saturday), you don't have to wait a week to see the free space. Only until Wednesday. (The exact time on Wednesday depends on when you added the file and when the snapshot was taken that INCLUDES that file.)

I need to clarify my answer.

If you take 2 daily snapshots, kept for 7 days, then yes, you have to wait a week. That's because the deleted file will exist in more than 1 snapshot.

In the case of adding a file on Wednesday, deleting on Saturday, you have to wait til next Saturday to see the space used by that file freed up. The file will be covered by Thursday's and Friday's snapshots. And possibly Wednesday and Saturday, depending on your snapshot schedule.


It's a bit hard to keep track of things like this. I make mistakes in describing these things.