Beginner's Hypothetical Question

[CPCJim]

I understand the basic minimum and preferred requirements of 1Gb of RAM per 1TB of storage. My question is hypothetically speaking, 2GB of RAM is needed for a RAID 0 with 2 @ 1TB drives, does it take 1GB or 2GB for RAID 1 with 2 @ 1TB of drives? I'm looking at building a system with very high capacity using an Intel board with 128GB or 256GB of RAM with Enterprise Level drives and am doing tests on some existing hardware using 4 @ 4TB drives in various RAID configurations. I've browsed the manual and can't seem to find the answer, or at least a straight answer, to the above question. Thanks!

 

[dlavigne]

There is no definitive answer, only rough guidelines, as it is highly dependent on workload and one's definition of acceptable performance. Best bet is to throw in as much RAM as possible and test the hardware and configurations before putting into production, so it sounds like you're on the right track

 

[Ericloewe]

The thumbrule is deliberately vague. More is better, but 16GB is enough for most small servers and 8GB is the absolute minimum.

 

[bestboy]

And from what I gather from users with big boxes here on the forum, it seems to be possible to substitute supplement RAM with SSDs for read workloads once a certain level is reached.
E.g. when a system has 128 GB of RAM, it might be ok to use a fast SSD as L2ARC instead of adding another 128 GB of RAM to the ARC.

For write workloads you need at least: your maximum, combined data ingress * the transaction group timeout
E.g. for 3x 10GbE NICs with a default txg timeout of 5 seconds you need: 3 * 1250 MB/s * 5 s = 18750 MB

 

[cyberjock]

Just to clarify, it's not a substitution. It's a supplement.

Keep in mind that with 3x10GbE NICs your math is right, but you'd also need slog devices capable of those speeds, which often means multiple devices that are all 4-figure numbers... to start. At some point you have to juggle "how far" you want to go. :P

 

[bestboy]

Just to clarify, it's not a substitution. It's a supplement

Noted.

Keep in mind that with 3x10GbE NICs your math is right, but you'd also need slog devices capable of those speeds, which often means multiple devices that are all 4-figure numbers... to start. At some point you have to juggle "how far" you want to go. :p

I did not consider slog devices as they do not directly contribute to the RAM requirements. It's certainly true that they are mandatory for write heavy workloads and that they contribute significantly to the bill :)

CPCJim did not disclose the purpose of the machine yet. If the workload is mainly going to be reads, then it might even be possible to use no slog device and just use the transaction log on the data drives, because he's

[...] building a system with very high capacity [...]

Depending on how the array is going to look like, it might even be ok to log the occasional input of the 3x 10GbE NICs on the data pool.

 

[gpsguy]

Given how much the equipment will cost, you might want to contact iXsystems and see they'd suggest.

If you go the DIY route look at
Supermicro instead of an Intel mobo.


Sent from my phone

 

[HankC]

so you can put multiple SLOG devices? more than 2?

 

[Ericloewe]

Of course, since they should be mirrored, at least.

 

[depasseg]

Yes you can have multiple SLOG and Cache devices. In my testing, multiple SLOG devices were automatically configured to be mirrored and multiple Cache devices got striped.

Your SLOG doesn't need to be big, just fast at writes.

 

[depasseg]

Of course, since they should be mirrored, at least.

I thought the mirror requirement wasn't as critical, due to a recent change.

 

[Ericloewe]

I thought the mirror requirement wasn't as critical, due to a recent change.

It won't destroy the pool, but whatever was in the ZIL is lost on device failure. That's bad enough for most people who decide "Well, better not disable sync writes...".

 

[CPCJim]

I would like thank all of you for your responses, they are all very helpful. I'm doing offline tests for our needs which are split by certain read heavy and other write heavy applications. I'm doing an offline test to "learn" what I may need and verify my understanding. Once I'm comfortable with my beginner's level of knowledge, I plan on a limited live test for each scenario. I'm seasoned enough to know there will be something I may miss, but you have greatly helped me decide I may actually need at least two separate setups, whether they're in the same box or different, I'll learn as I go. When I do decide to go "all out" I will contact iXsystems for a quote and may just go TrueNAS if the budget is allows. I like the SSD as L2ARC and the SLOG device ideas are just the type of knowledge I was looking for and as an old friend used to say "mo RAM solves a lot of ills and if it don't, add mo anyway, never heard anyone say, I just got too much RAM!" Was already considering the SSD for L2ARC. The screenshot and knowing SLOG devices are mirrored and Cache devices are striped is the type of knowledge I needed. Need to shore up my understanding of SLOG devices and thier role. Nearly lost 15 years of data on a NAS device with old un-upgradable firmware due to Apple's implementation of SMB destroying the old AFP Resource Forks! You guys are AWESOME - THANKS!