DeDupe NFS File Server Share

[RegularJoe]

Hi All,

I am wondering if we use a data set for NFS and it is a file server for all the users if we could identify duplicate files via ZFS. I know we should see a dedupe ratio but can we see the NFS file as well?

If not I will just look at the dedupe status and use a Linux box running fdupes.

Thanks,
Joe

 

[bigphil]

No. You're not going to actually see which files a deduped by ZFS...just the dedupe ratio. Also remember that there is a cost to using dedupe. In this case, its mostly RAM as the dedupe tables are held there. You're unlikely to see a good dedupe ration for user files. VDI deployments could see a benefit by using dedupe. The general consensus is to keep dedupe off and use the default compression algorithm instead.

 

[RegularJoe]

I guess I have to file a bug/suggestion to get the fdupes package added to FreeNAS so I can figure out what the users are doing without adding a VM or making a remote NFS machine to chase files.

 

[bigphil]

I guess I have to file a bug/suggestion to get the fdupes package added to FreeNAS so I can figure out what the users are doing without adding a VM or making a remote NFS machine to chase files.

Don't file a bug report for this...its not a bug. You can use a jail to install the package. https://forums.freenas.org/index.php?threads/installing-ports-such-as-rmlint-fdupes.23046/

 

[RegularJoe]

I guess it really would be an enhancement.

For FreeNAS to be used as a file server or a replacement for a commercial NAS some of these features need to be there. Users with unstructured file servers eventually make a mess and have stuff all over. To reduce the dedupe footprint it might be beneficial to hunt down these duplicate files and educate the user that is the owner of the file(s).

Thanks,
Joe

 

[MatthewSteinhoff]

@RegularJoe, do you find fdupes helpful? Do you use it as part of automatic processing or as a manual check for duplicates? How have you implemented fdupes?

Every time I looked at fdupes, it seemed more trouble than it was worth, often causing problems.

Cheers,
Matt