112 TB Server with only 32GB of RAM?

[christopher.blaisdell]

I have replaced ten of the fourteen 4tb drives in my array with 8tb drives. My server always seems stable and fast to me, but I notice that I am considerably out of step with the 1 GB per 1 TB guideline.

The problem is that my motherboard is maxed out at 32GB of RAM.

Should I complete my plan to replace all 14 drives? Or must I buy a new motherboard, processor, and memory first?

 

[m0nkey_]

I think you'll be okay. According to your signature it's a home server, so I doubt it's being hit very hard.

 

[christopher.blaisdell]

Thanks m0nkey, yeah, it's pretty much just me hitting it.

Am I at least doing the math correctly for the guideline? Is it 1 GB of Ram for every 1TB of physical drive space? Or 1 GB of RAM for every 1TB of usable drive space?

Also, is there a different guideline for Z2/Z3 versus a Raid10 (mirrored vDevs)? Shouldn't mirroring and striping require considerably less processing and memory than calculating ZFS parity bits?

And finally, do you think am I just risking slower performance? Or might I loose all my data?

 

[Ericloewe]

Is it 1 GB of Ram for every 1TB of physical drive space? Or 1 GB of RAM for every 1TB of usable drive space?

Yes.

It's deliberately vague.

less processing

Trivially so.

memory

No. Maybe a few MB? Trivial, at worst.

 

[christopher.blaisdell]

Yes.

It's deliberately vague.

Thanks for the clarification.

 

[Evi Vanoost]

Mirrors are faster than RAIDZ2 but not necessarily because of processing but because you get more VDEVs which increases simultaneous read/writes.

As far as memory, 1GB/TB is bollocks unless you have dedup on and then it can vary from ~1-4GB/TB depending on your data and setup. Having L2ARC requires a bunch of RAM. There used to be some excellent calculations on the Sun site, but with a bit of dtrace you can probably find out what your RAM is being used on.

 

[tvsjr]

Do it. And if you're just storing media, go RAIDZ2/Z3 and be happy. Worst case, you won't get optimum performance because you don't have a ton of memory for ARC. But you aren't at risk of losing data.

 

[christopher.blaisdell]

Worst case, you won't get optimum performance because you don't have a ton of memory for ARC. But you aren't at risk of losing data.

Thanks tvsjr, that's just what I was hoping to hear, and that makes sense to me.

And if you're just storing media, go RAIDZ2/Z3 and be happy.

Thanks tvsjr, I will consider that for Blaisdell Freenas server 2.0. My main concern would be the ease and time it takes to add storage & resilver for an array as large as 100TB+. I have a tendency to buy and add drives over time rather than feeling the pain all at once.

with a bit of dtrace you can probably find out what your RAM is being used on.

Vanoost, thank you, figuring out how to examine freenas memory usage was one of my goals in coming to this forum. I will dig in to dtrace.

 

[Stux]

You're aware that you can't add drives to a raidz vdev right?

 

[christopher.blaisdell]

You're aware that you can't add drives to a raidz vdev right?

Hi Stux, No, I had forgotten about that since deciding upon mirrored vdevs a year or two back. Thanks for the reminder.