Storage Usage seems significaly higher than it should be.

[nadinio]

Hey all,

I just got my first FreeNAS box deployed in my home and migrated all my things and such to it, but it seems to be occupying more space than it should. I've attached a screenshot showing that there is some how 7.7TiB of usage in my master data set, and 5.6TB of usage across all it's slave datasets. However the actual file size of everything stored is just over 4TB. What gives??

 

[SweetAndLow]

So the first line 'operations' represents the total usable space(zfs list Operations) in your pool. So this is 10.21TiB. The available column is always going to be the same because that is just free space and that is the same for all datasets in your pool. Then each dataset has the value of the data in that dataset and then the root dataset 'operations' should show the total of everything including the child datasets. At the very top next to 'HEALTH' this number is the raw size of your volume(zpool list Operations).

Sometimes people have snapshots configured and those take up space since they reference data that has been deleted. If this happens the freespace doesn't actually get free'ed until the snapshot is deleted. Do you have snapshots setup?

 

[nadinio]

It appears I have some snapshots but they only refer to about 2GB of stuff. I haven't really set these up yet.

I ran that zfs list command and saw that there is a 1.39TB reference which seems to add up to the "missing" storage space.

 

[SweetAndLow]

well running zfs list on your 'operations' dataset only shows you the refer for that dataset not all datasets. Everything looks normal to me. Do you still think there is a problem?

 

[nadinio]

I do think there is still a problem.

Used + Avail = 10.21 TB, this makes sense.
Total in use = 5.60TB
Total spaced used by files = 4.21 TB.
Total used by reference = 1.39TB (?)

I understand that the 4.21TB usage + 1.39TB Reference yields the 5.60TB usage it is reporting, I guess I am just lost on why there is an extra 1.39TB of space consumed. Is this like a ZFS overhead or something?

 

[SweetAndLow]

no it's not overhead. you sure you only have 2 snapshots? what is the output of zfs list -t snapshot?

 

[nadinio]

Output of snapshots looks pretty similar.

 

[SweetAndLow]

I do think there is still a problem.

Used + Avail = 10.21 TB, this makes sense.
Total in use = 5.60TB
Total spaced used by files = 4.21 TB.
Total used by reference = 1.39TB (?)

I understand that the 4.21TB usage + 1.39TB Reference yields the 5.60TB usage it is reporting, I guess I am just lost on why there is an extra 1.39TB of space consumed. Is this like a ZFS overhead or something?

This sentence at the end confused me. You have 5.60 worth of data on this system. And you have 4.61 free space. There is no 1.30TB reference.... The refer in the zfs list output just notes how much data that snapshot points at, that number does not affect any storage capacity number. The 'used' field in the snapshot list does represent some storage being used by that snapshot, for exmaple your 2 snapshots each use 488k worth of space.

 

[nadinio]

Maybe I am misunderstanding something. These other data sets are the only thing stored within the operations dataset. However, these datasets only add up to 4.21 TB. Wouldn't this leave me with 6.00 TB free?

 

[SweetAndLow]

Maybe I am misunderstanding something. These other data sets are the only thing stored within the operations dataset. However, these datasets only add up to 4.21 TB. Wouldn't this leave me with 6.00 TB free?

Are you sharing your root dataset? what is that .windows? DO NOT SHARE YOUR ROOT DATASET EVER!

ok now back to the original question. Yes you are correct all those datasets should add up to that 5.6TB number. I have no clue why it's not doing that. Especially since you don't have any snapshots. Do you have quotas or reserved space for anything?

 

[SweetAndLow]

What is the full output of zfs list?

 

[nadinio]

I think the .windows folder has something to do with the Windows ACL type that is set. Only the subdirectories inside Operations have a share, nothing to the root Operations directory.

Took a look through each dataset, no quotas are set.

Attatched output of zfl list. Used is 5.60TB but doesn't look like it adds up still.

 

[SweetAndLow]

I'm starting to think this could just be overhead. It just seems like to much.

 

[nadinio]

Probably should have mentioned this earlier but I didn't think much of it until now.

When first setting up the freenas server I just created directories instead of datasets. The more I learned, I decided to move over to datasets. I first created one on top of my Backups directory which I quickly learned doesn't link folder to dataset but just obliterates the directory while putting the dataset in. Oh well, it was all backups anyways, take another one, move on. Maybe it has something to do with this "overwritten" directory?

 

[SweetAndLow]

OMG this is it, the backup directory still exists and probably still has data in it. And since it's part of the root dataset it's totals get added to the root total.

 

[nadinio]

Oops.

Is there anyway to remove that folder or should I just drop the dataset, take another backup and use this as a learning experience?

 

[SweetAndLow]

you should be able to

Code:

zfs unmount Operations/Backups

delete the old Backups directory, double check it's actually what you want to delete.
Then run:

zfs mount Operations/Backups or if that doesn't work you can zfs mount -a

 

[nadinio]

That worked perfectly! Thank you so much for your help and patience!