IceBoosteR
Guru
- Joined
- Sep 27, 2016
- Messages
- 503
Hello guys,
I got a quick question about the FreeNAS ZFS deduplication integration. I spend some time and searched for this on the internet, but there was no suitable result.
So, I have one specific dataset with 400Gigs of data, where I have deduplication enabled (DD table is kinda small, 700 MB...). I have set deduplication on "ON" via the GUI. I know there is one function called "VERIFY" where every block which is identical per checksum-hash, is also compared byte-by-byte to avoid any problem, that the blocks are not identical.
So I was thinking about hash-collision. The documentation said, hash-collisions are rare these days, so there is no need for the verify option, only if you are kind of paranoid. So as there are dozen of hash algorithms existing, which one is openZFS using for block comparison by default, when doing this via GUI? Is it fletcher4? Somewhere I saw, that SHA-256 could be enabled, but only via shell, so I assume this is optional.
Can anyone be so kind and answer that question to me? I am really interested in this.
Another question on this, if I may ask:
What is happening when a hash-collision happens? So ZFS thinks the blocks are identical, but they aren't? What happens to the file, when I want to open it? Does the system reconize this time that the checksum is bad and does the self-healing from mirror, or is it not detected and you can end up with an corrupt file?
Because that would be the point, where I would use my extra horsepower from the Xeon for the verify-function, as I want also for this data maximum integrity.
Thank you in advance.
IceBoosteR
I got a quick question about the FreeNAS ZFS deduplication integration. I spend some time and searched for this on the internet, but there was no suitable result.
So, I have one specific dataset with 400Gigs of data, where I have deduplication enabled (DD table is kinda small, 700 MB...). I have set deduplication on "ON" via the GUI. I know there is one function called "VERIFY" where every block which is identical per checksum-hash, is also compared byte-by-byte to avoid any problem, that the blocks are not identical.
So I was thinking about hash-collision. The documentation said, hash-collisions are rare these days, so there is no need for the verify option, only if you are kind of paranoid. So as there are dozen of hash algorithms existing, which one is openZFS using for block comparison by default, when doing this via GUI? Is it fletcher4? Somewhere I saw, that SHA-256 could be enabled, but only via shell, so I assume this is optional.
Can anyone be so kind and answer that question to me? I am really interested in this.
Another question on this, if I may ask:
What is happening when a hash-collision happens? So ZFS thinks the blocks are identical, but they aren't? What happens to the file, when I want to open it? Does the system reconize this time that the checksum is bad and does the self-healing from mirror, or is it not detected and you can end up with an corrupt file?
Because that would be the point, where I would use my extra horsepower from the Xeon for the verify-function, as I want also for this data maximum integrity.
Thank you in advance.
IceBoosteR