Encryption/dedup/compression

[qlx309]

Hi All

I've been running my FreeNAS server for the last 6 weeks on 11.1-RC1 and it has been great. I upgraded to 11.1-RELEASE yesterday and it went well. My physical server running FreeNAS has the following hardware:

• Supermicro X10SL7-F
• Xeon(R) CPU E3-1230 v3 @ 3.30GHz
• 32GB DDR3 ECC RAM
• 4 x Samsung Enterprise 480GB SATA SSDs
• 4 x Samsung Enterprise 960GB SATA SSDs
• 2 x Samsung Pro 840 120GB SATA SSDs (boot)
  
• 500W Seasonic PSU
• Mellanox ConnectX2 - dual port

FreeNAS is used as shared iSCSI storage for my vSphere cluster.

I have two ZFS pools setup as follows:

• Striped mirror consisting of 4 x 960GB SSDs - total pool size is 1.8TB
• RAIDZ1 consisting of 4 x 480GB SSDs - total pool size is about 1.3TB

Total flash storage across both pools is just over 3TB but I am currently using about 900GB of this.

Because I run about 30 VMs that run the same version of Windows in my vSphere cluster I would like to try dedup. I also want to encrypt my ZFS pools. From what I have read I need to destroy the pools and recreate them which is fine. I have compression turned on for both volumes currently.

I know dedupe uses lots of RAM but do I have enough for my storage? I've read you need 5GB per TB so I shouldn't need more than 15GB out of the 32GB of RAM I have for dedupe. Do you work out how much RAM you need for dedup for the total size of the volume or just for the used space? I assume having about 17GB of RAM for the system/ARC is enough?

My CPU supports AES-NI so will there be any issues with encrypting both of my volumes?

Ultimately I would like to have compression and dedup turned foe each volume AND have encryption enabled.

Thanks!

 

[Ericloewe]

Because I run about 30 VMs that run the same version of Windows in my vSphere cluster I would like to try dedupe.

Trust me, you don't. Instead, install Windows and then clone the dataset as many times as necessary. It won't help with updates, though, unless you repeat the process regularly.

Do you work out how much RAM you need for dedupe for the total size of the volume or just for the used space?

The ZFS books describe a method you can use to size the RAM you need for deduplication.

I also want to encrypt my ZFS pools.

Why? Unless the answer is "I have a legal/contractual/other bureaucratic requirement", you really shouldn't.

My CPU supports AES-NI so will there be any issues with encrypting both of my volumes?

You're far more likely to lose your data, but it's otherwise fine. Though at SSD speeds, the CPU might struggle to keep up.

 

[garm]

I believe the rule of thumb for RAM/deduplication is slightly misleading. I don’t remember the numbers, you might want to consider buying the ZFS manual, but the issue is that the deduplication table need to fit in memory. So you need to ensure that you control the amount of data being deduplicated and the RAM available. You can’t have the data grow over time for instance, becasue you will run out of space for the deduplication table and then your in for a world of hurt.

 

[qlx309]

I believe the rule of thumb for RAM/deduplication is slightly misleading. I don’t remember the numbers, you might want to consider buying the ZFS manual, but the issue is that the deduplication table need to fit in memory. So you need to ensure that you control the amount of data being deduplicated and the RAM available. You can’t have the data grow over time for instance, becasue you will run out of space for the deduplication table and then your in for a world of hurt.

My total space is 3TB and will never grow past that so 3TB is the maximum size which requires about 15GB of RAM...am I ok with 32GB of total memory in the server?

Trust me, you don't. Instead, install Windows and then clone the dataset as many times as necessary. It won't help with updates, though, unless you repeat the process regularly.

The ZFS books describe a method you can use to size the RAM you need for deduplication.

Why? Unless the answer is "I have a legal/contractual/other bureaucratic requirement", you really shouldn't.

You're far more likely to lose your data, but it's otherwise fine. Though at SSD speeds, the CPU might struggle to keep up.

Why don't I want to use dedup IF I have enough memory? Not saying you're wrong just looking for the pros and cons. Not sure I follow what you mean by "clone the dataset" many times.

I encrypt everything at home. Yes, I know and understand the risks (backup the key, document the passphrase etc etc). So why shouldn't I encrypt then if I have taken the necessary precautions to encrypt ZFS?

When I was using another trial iSCSI product I had compression and dedup enabled for my vSphere environment and it saved me loads of space which is what I am hoping I can do with FreeNAS!

 

[Ericloewe]

Why don't I want to use dedupe IF I have enough memory?

It's extremely slow and "enough memory" isn't as simple as the rule of thumb for the minimum makes it sound.

Yes, I know and understand the risks (backup the key, document the passphrase etc etc).

You're missing "backup the GELI metadata".

 

[wblock]

As @Ericloewe suggests, cloning is far more efficient than dedup and takes much less RAM: http://doc.freenas.org/11/storage.html#deduplication.

 

[qlx309]

Thanks for all the helpful replies.

I wanted to run:
zdb -b POOL_NAME

to test dedup but I get this error:

Code:

zdb: can't open 'RAIDZ1_480/RAIDZ1_480': No such file or directory

I also tried zdb -S but that also gives the above error.

What is the correct command to run to test/simulate dedup?