praecorloth
Contributor
- Joined
- Jun 2, 2011
- Messages
- 159
So I was experimenting with deduplication last night. Just doing some tests to see what kind of CPU and memory overhead I could expect. Unfortunately I couldn't get dedup to do anything. So here's what I did.
I fired up a fresh install of FreeNAS on a server with a pair of quad core Xeons and 8GB of memory. Not a ton of memory, but should be enough for some function testing. Made a RAIDZ of three 250GB SAS drives and set up a Samba share. I copied over a few video files which came out to about 11GB. I turned deduplication on and was watching the CPU usage to see if it was doing anything. It was not.
So then I made a new directory and copied the same 11GB of data. No extra CPU usage. No extraordinary amount of memory usage. When the copy was done, about 22GB of space was used. So I deleted the extra directory, and then rebooted the FreeNAS machine in case it needed a reboot to enable dedup.
I copied over the same 11GB of data, with the same result. I created a third directory and copied the same data. Still no extra CPU usage, no extra memory usage, and 33GB of space used on the share.
So while doing all of this, my expectations were that I would see CPU and memory usage that was out of the ordinary, and that when running a df -h against the RAIDZ, that only the space actually used would appear. Am I incorrect in my expectations? What should I be seeing with dedup on, and how can I test to make sure it's working properly?
I fired up a fresh install of FreeNAS on a server with a pair of quad core Xeons and 8GB of memory. Not a ton of memory, but should be enough for some function testing. Made a RAIDZ of three 250GB SAS drives and set up a Samba share. I copied over a few video files which came out to about 11GB. I turned deduplication on and was watching the CPU usage to see if it was doing anything. It was not.
So then I made a new directory and copied the same 11GB of data. No extra CPU usage. No extraordinary amount of memory usage. When the copy was done, about 22GB of space was used. So I deleted the extra directory, and then rebooted the FreeNAS machine in case it needed a reboot to enable dedup.
I copied over the same 11GB of data, with the same result. I created a third directory and copied the same data. Still no extra CPU usage, no extra memory usage, and 33GB of space used on the share.
So while doing all of this, my expectations were that I would see CPU and memory usage that was out of the ordinary, and that when running a df -h against the RAIDZ, that only the space actually used would appear. Am I incorrect in my expectations? What should I be seeing with dedup on, and how can I test to make sure it's working properly?