mv command filling drive?

[oguruma]

I used the mv command to move the contents of a directory into a different directory. The reason being is that I erroneously made a directory in Windows directly under the volume. So I created a dataset and use mv /* to move the contents into the dataset.

The problem is that this process has completely filled my drive.... My assumption was that the mv command would not result in an increase in the amount of storage used. Before I executed the command, I had roughly 2TB remaining on the drive. I woke up this morning with only 32MB remaining, and the move process has stopped.

The old directory shows 3.36TB being used, and the new one shows 1.8TB being used. On an 8TB drive (formatted to 7.2TB) that means I should have the the roughly 2TB left that I started with.

So, the question is: what happened to those 2TB?

 

[Jailer]

Sounds like your drive filled up before the move could be completed.

 

[oguruma]

Sounds like your drive filled up before the move could be completed.

It seems so, however it moves one file at a time does it not?

If the command moves the files one file at a time, what is occupying the "missing storage" space in the meantime?

I haven't been able to find a direct answer to this, but does mv write and then delete the "old" copy? If so, where does one find the "old" copy to permanently delete it?

 

[Jailer]

It does write and then delete but I don't know if that deletion takes place after the entire move is done or as each file is written. It looks as if it deletes after the move is complete based on your results.

 

[sretalla]

I can confirm from past experience that it seemed to keep all files in the origin until the end of the mv.

Maybe it's more a factor of the way that zfs handles the release of no-longer-needed storage to free space, but I've seen these same symptoms before.

 

[oguruma]

I can confirm from past experience that it seemed to keep all files in the origin until the end of the mv.

Maybe it's more a factor of the way that zfs handles the release of no-longer-needed storage to free space, but I've seen these same symptoms before.

Interesting... so any idea where are the quasi-deleted files? ls on both the source and destination directories makes it look like the deletion has occurred from the source with the same number of files in total...

How does one proceed from this point? Is there a way to tell ZFS to delete the files its been hanging on to?

 

[MrToddsFriends]

Snapshots?

 

[oguruma]

Snapshots?

To no avail. I rolled back to a snapshot from 3 days ago (I started the mv yesterday) and I still show the drive at 99% full. The UI indicated "rollback successful". The source destination has been rolled back to containing all of the files, but the disk is still 99% full (in its rolled-back state it should be about 70% full). The destination directory shows that the files have been removed, however the disk is doesn't indicate the same.

In other words, the rollback makes the directory appear as if it was rolled back, but there is still 2TB lost to the ether....

A reboot and another attempt at rolling back yields the same results.

 

[sretalla]

You'll need to delete all snapshots before the point where the mv started in order to get back the space.

A nice trick I learned recently is using the percentage symbol in-between two snapshots to denote a time period to delete snapshots from.

Something like:

zfs destroy tank/dataset1@Monday%Friday . Or whatever naming your snapshots have...

Note that in the above example, snapshots between the Monday and Friday ones (from a time perspective, not caring about names specifically) will be destroyed, including Monday and Friday themselves.

 

[pro lamer]

zfs destroy tank/dataset1@Monday%Friday . Or whatever naming your snapshots have...

Note that in the above example, snapshots between the Monday and Friday ones (from a time perspective, not caring about names specifically) will be destroyed, including Monday and Friday themselves.

Does it remove snapshots not caring what week they were? Does it delete all: the first, second, third Friday's of the year (and so on) snapshots?

 

[oguruma]

I deleted all of the snapshots to get to the one that I needed.

The rollback was successful. The issue I have is that the source destination was rolled back, but there is still 2TB of used space that I can't seem to account for....

 

[MrToddsFriends]

The issue I have is that the source destination was rolled back, but there is still 2TB of used space that I can't seem to account for....

Is this posting helpful for you?
https://forums.freenas.org/index.php?threads/snapshots-and-used-space.63810/#post-457224

Especially the exemplified
zfs list -r -o space,refer,written -t all <dataset>
command to see the values of the REFER and WRITTEN properties of a <dataset> (including snapshots, among other properties) and the
zfs destroy -n -v <range_of_snapshots>
command to see how much space would be freed by deleting a certain <range_of_snapshots>?

 

[sretalla]

Does it remove snapshots not caring what week they were? Does it delete all: the first, second, third Friday's of the year (and so on) snapshots?

To clarify, I probably should have used dates in my example to be more illustrative, but I did mention that you need to pay attention to what your snapshots are called.

So if you used:

zfs destroy tank/dataset1@20180101%20180131

That's all the snapshots in January 2018 gone for that dataset (assuming your snapshots are named according to the dates they were taken).

It doesn't matter if you took other snapshots manually in-between called @backup1 or @justbeforelunch, as long as they are sequentially between the two you specified, they will go.

The key to understand is sequential... ignore the names and just pick the snapshots that represent the first and last points in time you care about removing. Everything chronologically between them is removed.

 

[oguruma]

Well, I was hoping for a more elegant solution, but I basically just destroyed datasets...

The crux of my issue was that apparently mv doesn't delete objects from the source until ALL of the objects have been moved.... Since there wasn't enough space to accommodate the process, it got hung up and I couldn't fix it any other way than to destroy the destination dataset. That's not so say there isn't another way, I just don't know what it is....