Using DeDuplication with almost no concern for performance

[dpearcefl]

I have a ton of files (100 TB total ) that dedup very well. If I am not concerned about performance (yeah, silly, I know) and am using a FreeNAS with 32 GB ECC of memory instead of the recommended 500 GB of memory, other than being slow (due to paging I assume), would this be considered a "fragile" setup, likely to blow up regularly?

Thanks.

 

[anodos]

I have a ton of files (100 TB total ) that dedup very well. If I am not concerned about performance (yeah, silly, I know) and am using a FreeNAS with 32 GB ECC of memory instead of the recommended 500 GB of memory, other than being slow (due to paging I assume), would this be considered a "fragile" setup, likely to blow up regularly?

Thanks.

It depends on how large your dedup table ends up being. The problem with deduplication is that if your dedup table becomes too large you won't be able to import your pool until you add more RAM. This might mean buying a new motherboard (assuming that your motherboard only supports 32GB RAM) and adding a boatload of RAM to get things working again.

 

[gpsguy]

Ensure that you have good backups. There may be a day when you don't have enough RAM to mount your pool.


Sent from my phone

 

[dpearcefl]

Like on a reboot?

So the dedup table will not use swap?

How do I determine the current size of my dedup table on my test box?

 

[anodos]

Like on a reboot?

So the dedup table will not use swap?

How do I determine the current size of my dedup table on my test box?

No it does not use swap. You shouldn't see swap usage on FreeNAS. Since you're already testing this, please realize that basically the only way to remove deduplication is to destroy the dataset for which you enabled deduplication. Regarding viewing size of dedup table, see this post: http://serverfault.com/questions/533877/how-large-is-my-zfs-dedupe-table-at-the-moment

 

[dpearcefl]

Thanks for everyone's wonderful help.