Size on Disk (ZFS 4k) data is 65% larger than Data size

[JW0914]

Does anyone know why 594GB of data (which registers as 594GB in size and size on disk in Windows) would equate out to 920GB as size on disk, an increase of 64.57% (FreeNAS does recognize it as 594GB in size, but 920GB as size on disk, an exorbitant amount of wasted space)?

Before transfer, the size of the folder on FreeNAS was ~440ish GB, now 1.4TB, with only 594GB of actual data transferred.

Folder properties on Windows, as well as the capacity utilized from FreeNAS (before transfer used space was ~440ish GB, now 1.4TB (copied to "Data"), with only 594GB of actual data transferred).

• Property Window 1 (External HDD; Windows):

Size: 594GB (638,060,674,332B)
Size on Disk: 594GB (638,856,667,136B)
Contents, Files:444,838 Contents, Folders: 39,383​

• Property Window 2 (Network Share; FreeNAS)

Size: 594GB (638,060,674,332B)
Size on Disk: 920GB (988,161,245,184B)
Contents, Files: 444,838 Content, Folders: 39.384
(1 extra empty folder)

• No snapshots have been taken (even if they were, that value has no correlation to the folder size; correlation goes the opposite direction)
• Each folder contains the exact same 444,838 files, with one showing the data size as 594GB yet the other shows the exact same 444,838 files as 902GB (FreeNAS)
  • Prior to data transfer, used space on the dataset in question was ~440GB, after file transfer 1.4TB, a difference of 900GB+
    • Since the 594GB within this folder was stored on a 750GB drive (698GB formatted), I'm certain the 444,838 files are indeed 594GB in size.
      • 920 - 594 = 326GB, or 64.57%

• Data is measured by two different formulas, but regardless of which formula you use, the data neither increases nor decreases in size. Since this folder containing 594GB of data (not compressed) resided on a 750GB HDD (~698GB formatted) with a 100GB file and ~3.5GB of space free, this irrefutably shows the data is indeed 594GB in size. Unix based systems do not compute data used the same way Windows based systems do, however, if a 750GB HDD only has ~698GB free once formatted and on that drive resides a folder containing 594GB of data, coupled with an additional 100GB file, leaving ~3.5GB of free space on the drive, it can be shown irrefutably the data in question is no larger than 594GB.

All of which brings me back to my question: Why would a zfs array (created with force 4k) register 594GB of data as 920GB, a 64.57% increase in size?

 

[dlavigne]

Did you figure this out?

 

[JW0914]

Nope.... although I have an idea

Windows, intelligently I might add, stores data in 512b logical sectors, while utilizing 4k physical sectors. I believe FreeNAS is storing data in either 1k or 4k logical blocks, which is wasting a crapload of space.

I forced 4k sectors when I created the zfs file system, however it appears FreeNAS or FreeBSD does not by default intelligently select 512b logical sectors. While it makes the most sense to utilize 4k physical sectors, as 8 512b physical sectors would fill up 2 4k sectors due to the size being larger than 4k with the ecc digit. However, while the reason for 512b logical sectors was backwards compatibility, it serves a greater purpose by preventing wasted space on an hdd. While 10 - 15% wasted space would be acceptable, though not ideal, a 65% ratio is ludicrous and ridiculous.

 

[cyberjock]

I can definitely tell you that your idea is not only kind of debunked, its also somewhat validated.

Your ashift value represents the smallest assignable block for user data. This is 12 by default, which translates to 4KB. So anything smaller than 4KB will take up 4KB. Additionally, if you are storing just 3.5KB, it will end up taking up more space for parity. For RAIDZ3, this means that if you have a single file that is 3.5KB of data will take up 4 blocks, or 16KB, because you have to store the actual file plus 3 parity blocks. That is over a 400% increase. Makes your 65% not look so bad, eh?

Anyway, to come back around full circle, depending on your zpool layout you can have more or less wasted space. Also things like snapshots, manually changing block sizes, and a boatload of other settings you can manually adjust can have a profound affect on the zpool.

In short, your numbers are definitely not what I'd want to see, and definitely not typical of what I see in my day-to-day operations. I can tell you that if you force a smaller maximum block size, you can see disk usage that is absurdly high. I'm talking MUCH more than the 65% you are seeing.

So without much more info on what buttons and tweaks you've done, you are kind of on your own to figure out what is going on. I can tell you that this is definitely not the norm, so I tend to guess that you've tweaked a setting somewhere without fully understanding its implications, and are now seeing the result of that change. Sorry I can't be more exact because the number of ways you could waste space is more than I can count on one hand, and there's no doubt more than that as I haven't even had my coffee yet this morning.

 

[JW0914]

Perhaps I have a misunderstanding of how zfsx configuration is set up filesystem wise... my understanding was the parity is done on the apportioned data that is set aside and not useable. I have 17TB of drives in zpool 1, with ~ 10TB of usable space... do the 3 parity copies not use the 7TB or so that zfs sets aside for parity?

As far as tweaks go, nothing filesystem wise has been tweaked, except for checking the box to force 4k sectors when I originally formatted the drives for zfs. If you can shoot me in the right direction on how I can determine if filesystem tweaks are in place due to some other configuration I did, I'd appreciate it.

EDIT: I didn't come across, until after a lot of data was already transferred, that I should have given a gparted command to start the file system at a block value that's aligned to 4k... could this be a contributing factor?

 

[cyberjock]

Parity is not "set aside" like it is for hardware RAID. It is created, on the fly, based on when the writes are made to the system, block size, etc.

EDIT: I didn't come across, until after a lot of data was already transferred, that I should have given a gparted command to start the file system at a block value that's aligned to 4k... could this be a contributing factor?

Ah, the truth is rearing its ugly head. You shouldn't be using the CLI for anything, and you should be sticking to the defaults. So yeah, you've basically created some frankenmonster, and not surprisingly it's not going well. So yes, rebuild the system, and use the WebGUI. If you have to use the CLI you are probably screwing up, and you are probably creating problems that aren't easy (if even possible) to fix later.

So when I felt that you were tweaking things, you not only tweaked them, you've literally done things that the manual doesn't have you do. You'll find that FreeNAS works better when you use it as designed.

 

[JW0914]

Parity is not "set aside" like it is for hardware RAID. It is created, on the fly, based on when the writes are made to the system, block size, etc.



Ah, the truth is rearing its ugly head. You shouldn't be using the CLI for anything, and you should be sticking to the defaults. So yeah, you've basically created some frankenmonster, and not surprisingly it's not going well. So yes, rebuild the system, and use the WebGUI. If you have to use the CLI you are probably screwing up, and you are probably creating problems that aren't easy (if even possible) to fix later.

So when I felt that you were tweaking things, you not only tweaked them, you've literally done things that the manual doesn't have you do. You'll find that FreeNAS works better when you use it as designed.

You continue to make a lot of assumptions that are not only blatantly wrong, they're arrogant and come across as so.

Thanks, but no thanks... please don't comment on any more of my posts... this is the third time you've repeatedly shown not just your arrogance, but blatant rude, disrespectful, and condescending behavior.

Instead of offering substantive information, you instead choose to offer nothing more that arrogance based off predicated assumptions there's no basis for.

In closing, I'll address your post:

Using gparted to format my drives didn't cause this issue, and it's ludicrous to even contemplate one formating their drives with gparted or disk part is wrong and it screws up things... newsflash... freenas uses gparted commands behind the gui.

A word of advice: grow up and act like an adult

 

[Ericloewe]

You continue to make a lot of assumptions that are not only blatantly wrong, they're arrogant and come across as so.

Thanks, but no thanks... please don't comment on any more of my posts... this is the third time you've repeatedly shown not just your arrogance, but blatant rude, disrespectful, and condescending behavior.

Instead of offering substantive information, you instead choose to offer nothing more that arrogance based off predicated assumptions there's no basis for.

In closing, I'll address your post:

Using gparted to format my drives didn't cause this issue, and it's ludicrous to even contemplate one formating their drives with gparted or disk part is wrong and it screws up things... newsflash... freenas uses gparted commands behind the gui.

A word of advice: grow up and act like an adult

You'll find that "please don't comment on any more of my posts" will be parsed by everyone as "I have no desire to engage in a meaningful discussion.", especially if it was the answer to the explanation of what you did wrong (not using the GUI).

 

[JW0914]

You'll find that "please don't comment on any more of my posts" will be parsed by everyone as "I have no desire to engage in a meaningful discussion.", especially if it was the answer to the explanation of what you did wrong (not using the GUI).

If believing that means I don't have to deal with his b.s., then by all means, believe whatever you want

 

[zambanini]

jw will be on many ignore lists. well done jw.

 

[cyberjock]

He is also banned. He was given a warning 2 weeks ago by someone else. So nobody need to worry about him again.

Locking the thread.