I'm just starting out with a new FreeNAS setup. I've gotten set up to the point where I have CIFS sharing working and I can copy files onto the NAS from clients.
Now I want to transfer a large amount of data to the NAS. Is there a good way to verify that all the data was copied correctly? I know ZFS has block level checksumming and that will make sure everything that gets to ZFS is written to disk correctly, but how can I make sure no corruption happens earlier? Does CIFS detect corruption or use any FEC scheme?
The most obvious thing to me seems to be to hash every file on both the source and destination machines, write those to a text file, and then check that the file has the same list of hashes. I have a few issues with that approach:
1. It's a lot of effort.
2. The source is Windows and the destination FreeNAS, so I have to make sure whatever tools actually get the same hashes for the same logical set of files ignoring whatever hidden stuff gets put into the respective filesystems.
3. The FreeNAS volume is much bigger than anything on the source, so I'm moving from multiple source directories into a single destination. So a single recursive statement will hit everything in FreeNAS, but the source will need multiple, which means additional processing to get all those hash comparisons "aligned". Doable, but likely a PITA.
4. I don't actually have a good idea of what file hashing tools are available on either platform.
Any advice?
Now I want to transfer a large amount of data to the NAS. Is there a good way to verify that all the data was copied correctly? I know ZFS has block level checksumming and that will make sure everything that gets to ZFS is written to disk correctly, but how can I make sure no corruption happens earlier? Does CIFS detect corruption or use any FEC scheme?
The most obvious thing to me seems to be to hash every file on both the source and destination machines, write those to a text file, and then check that the file has the same list of hashes. I have a few issues with that approach:
1. It's a lot of effort.
2. The source is Windows and the destination FreeNAS, so I have to make sure whatever tools actually get the same hashes for the same logical set of files ignoring whatever hidden stuff gets put into the respective filesystems.
3. The FreeNAS volume is much bigger than anything on the source, so I'm moving from multiple source directories into a single destination. So a single recursive statement will hit everything in FreeNAS, but the source will need multiple, which means additional processing to get all those hash comparisons "aligned". Doable, but likely a PITA.
4. I don't actually have a good idea of what file hashing tools are available on either platform.
Any advice?
