16GB or 32GB RAM for 5X8TB?

[rangopango]

Would I really need 32GB RAM for five 8TB drives in Raidz2? I heard the general consensus is to have 1GB of RAM for each TB, but is that for usable space on the drive?

 

[Chris Moore]

Would I really need 32GB RAM for five 8TB drives in Raidz2? I heard the general consensus is to have 1GB of RAM for each TB, but is that for usable space on the drive?

It is a general guideline to help with memory sizing because FreeNAS uses your system memory for something called ARC (Adaptive Replacement Cache) which allows the system to provide frequently accessed files from memory very quickly instead of needing to read them from disk every time.
If you don't have enough memory, performance may suffer, but the system should be perfectly stable with that much storage and 16GB of RAM.
Total space in the array should be around 36TB, but usable space will only be about 20TB and taking into account the 80% limit, you should try to keep usage under about 16.5TB. Rough estimates.

 

[rangopango]

Thank you for the detailed answer. Two more question, if I plan to use compression, would it be a good idea to use more than 16GB RAM? I'm planning on using compression for mostly 1TB~ of pictures while the rest remains uncompressed. Is it a bad idea to turn off ARC? I don't really care about cache... and I don't plan on accessing files with a low file size regularly.

 

[Chris Moore]

if I plan to use compression, would it be a good idea to use more than 16GB RAM? I'm planning on using compression for mostly 1TB~ of pictures while the rest remains uncompressed.

Just compress it all. The default compression on FreeNAS is lz4 and there is very little overhead. I use it for everything and it is totally worth doing. I don't understand why you would think of turning compression off. If I recall correctly it is on by default.

Is it a bad idea to turn off ARC? I don't really care about cache... and I don't plan on accessing files with a low file size regularly.

ARC is part of ZFS. I would never consider turning it off. Why would you think this is beneficial. Is someone suggesting this to you?
There is no situation I am aware of where turning ARC off would be anything but bad.

 

[rangopango]

Nope, I just assumed it for a silly reason. Isn't it unnecessary to use compression for video files that's already been compressed though?

 

[Chris Moore]

Nope, I just assumed it for a silly reason. Isn't it unnecessary to use compression for video files that's already been compressed though?

It isn't necessary but it doesn't hurt anything either.
There's no reason to segregate the data out in the different data sets which would be necessary in order to have different compression

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Chris Moore]

Nope, I just assumed it for a silly reason. Isn't it unnecessary to use compression for video files that's already been compressed though?

Here is some documentation that you might want to review to get more familiar with the concepts in ZFS and FreeNAS:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

 

[wblock]

Nope, I just assumed it for a silly reason. Isn't it unnecessary to use compression for video files that's already been compressed though?

ZFS uses compression for each block. If the compressed version is not any smaller than the original, it is not used. So there really isn't any penalty.