NTFS vs ZFS Disk Usage

[sjmorrow]

Hi

I'm new to FreeNAS and this whole ZFS thing. I have all my data on two NTFS drives mirrored using my Nvidia ION onboard raid "thing" (because its not really a controller...), and am working on migrating all that data off of them so I can do away with the FakeRAID and format them to ZFS for use in FreeNAS. So I created a zpool using some spare drives I had laying around, mounted my ntfs mirror in FreeNAS (using ntfs-3g and -ro option), and rsync-ed all my files on the NTFS drives to the zpool.

My question is why is the disk usage on my zpool 823GB, but the disk usage on my NTFS drive is 904G? I am terrified that something didn't get copied over, so I haven't reformatted my NTFS drives yet. I tried running rsync multiple times (with rsync -avh --modify-window=604800 /mnt/Shares /mnt/NFSBackup , to avoid the problem with NTFS timestamp conversion, and because I havnt' touched the files on these drives in a month or more)

 

[TheSmoker]

What version of freenas are you using? I am asking this because in the latest version, when creating a new volume it will enable compression by default. That's where the size difference might come from.

 

[survive]

Hi sjmorrow,

I bet you are seeing the standard "reserves 10% of the file space" thing that most Unixes do. For example they report 900GB when you have a terabyte to try to keep you from filling up the file system.

There's a tool I like called "Beyond Compare" that's really handy for comparing drives, directories & files between 2 file systems....it might be worth grabbing a copy and double check that everything is moved over the way you want.

-Will

 

[sjmorrow]

@TheSmoker: yes I am using the latest, so yes compression is enabled. The thing that troubles me is if you divide the NTFS file size by the ZFS file size, you get a 1.098 compression ratio and the FreeNAS GUI reports only 1.07 compression ratio.

@survive: thanks very much for the response. I'm going to go google "Beyond Compare" right now.

 

[Richman]

Hi sjmorrow,

I bet you are seeing the standard "reserves 10% of the file space" thing that most Unixes do. For example they report 900GB when you have a terabyte to try to keep you from filling up the file system.

I think regarding what you are pointing out has to do with 'alocated available space'. What he is talking about is is amount of 'datawritten' to the drive.

There's a tool I like called "Beyond Compare" that's really handy for comparing drives, directories & files between 2 file systems....it might be worth grabbing a copy and double check that everything is moved over the way you want.

I was going to mention such a tool but couldn't think of a specific name. Does this tool auto search and point out differences?

-Will[/quote]

@TheSmoker: yes I am using the latest, so yes compression is enabled. The thing that troubles me is if you divide the NTFS file size by the ZFS file size, you get a 1.098 compression ratio and the FreeNAS GUI reports only 1.07 compression ratio.

Maybe this is similar to the 'Size' versus 'Size on Desk' that widows reports.

 

[cyberjock]

Uh.. those compression ratios are complete crap...

https://bugs.freenas.org/issues/4030

And that's the default because nobody wanted to back me up.

You're welcome. ;)

 

[indy]

LZ4 default on seems like a reasonable standard setting for most zfs capable hardware.
Compression on usual home-server stuff like media files is basically zero though.
The algorithm is supposed to abort compression early on these files anyway however.