Since I started using Freenas, I have made changes to my folder and dataset structure to make more efficient usage of the snapshot capability of ZFS.
I have been moving files and directories across datasets and because of the snapshots, I cannot recover the space unless I destroy snapshots prior to the data migration.
I have been looking around for ways to check differences between folder but this solution doesn't seem to be very efficient.
I have found mention of the "diff" command and the like, but this is limited to having to compare identical datasets, folders or files.
I am trying to push the search further and today I came to realize I can use CRC check capability within Freenas and scan the entire pool, the result being placed into a file for further analysis.
the command I am using is:
This is going to be a fairly lengthy process, but this as the advantage to being run within Freenas itself using SSH under a "screen" or "tmux" session.
This may or may not be optimal, but I am looking at a way to break the "find" output across multiple threads/cores.
For instance, if "find" goes through a folder with 10 files, ie file_1, file_2, file3...., file_10, for each file it will run one instance of shsum, waiting for each instance being completed before moving to the next file.
Is it possible to set "find" to execute file_1 on one instance of "shsum" then file_2 on a second instance of "shsum" so that multiple instance of "shsum" running on different file can be run simultaneously, then when the first instance is finished, then "find" will send the next file file_3 to the first instance and so on until "find" doesn't have anything else to process?
I have been moving files and directories across datasets and because of the snapshots, I cannot recover the space unless I destroy snapshots prior to the data migration.
I have been looking around for ways to check differences between folder but this solution doesn't seem to be very efficient.
I have found mention of the "diff" command and the like, but this is limited to having to compare identical datasets, folders or files.
I am trying to push the search further and today I came to realize I can use CRC check capability within Freenas and scan the entire pool, the result being placed into a file for further analysis.
the command I am using is:
Code:
find /mnt/my_pool/ -exec shasum -a 512 -b {} \;>> shasum_My_pool.txt
This is going to be a fairly lengthy process, but this as the advantage to being run within Freenas itself using SSH under a "screen" or "tmux" session.
This may or may not be optimal, but I am looking at a way to break the "find" output across multiple threads/cores.
For instance, if "find" goes through a folder with 10 files, ie file_1, file_2, file3...., file_10, for each file it will run one instance of shsum, waiting for each instance being completed before moving to the next file.
Is it possible to set "find" to execute file_1 on one instance of "shsum" then file_2 on a second instance of "shsum" so that multiple instance of "shsum" running on different file can be run simultaneously, then when the first instance is finished, then "find" will send the next file file_3 to the first instance and so on until "find" doesn't have anything else to process?