BUILD Recommended Memory - Home Server Use

[Scharbag]

I have read many different posts on memory needs for ZFS, such as the 1GB per TB etc. For a home system, where performance is not crucial, what is would be the MINIMUM amount of ram that will work for the following:

• ~30TB production pool
• ~25TB backup pool
• ~500GB SSD based pool (Jails and VMs)

I ask this because my Supermicro server that I bought (dual Xeon, 96GB ECC) is just too loud as I do not have a cool out-of-the-way bunker to put it in. I still have my old system with an E3-1230-V2 CPU and 32GB of ECC ram. Due to storage growth requirements, I need to purchase another Norco 4224 case for my additional HDD requirements.

I also virtualize some Windows machines and run a few jails in 9.10. I have not tried Corral or ioHyve with virtualization yet. Most of what I run is very lightweight and not CPU intensive. See CPU history below (cause I have them, I gave FreeNAS 8 vCPUs):

My old system used to run fine when I would virtualize with VirtualBox. Now that VirtualBox is deprecated, I utilize ESXi as I have not figured out how to use ioHyve or BHyve. That is next on my list.

Long story short, will 32GB be enough to FreeNAS and ESXi?

Thanks,

 

[snaptec]

That really depends on your load.
32gb and 100tb+ can work without problems.
16gb RAM with 4TB as vm storage can feel very laggy.
The more ram, the more performance, depending on load.
I'm sorry, but you have to try if you are happy with it.


Gesendet von iPhone mit Tapatalk

 

[Scharbag]

Yeah, my system loading is very minimal, mostly HTPC content serving. I have some windows VMs that I need but they do very simple things, such as accounting software and metadata management.

I will likely standup my old server and install Corral on it and try to figure out all the ins and outs of Docker and BHyve so that I can do away with the ESXi layer.

Cheers,

 

[Dice]

IIRC, ~ "I wouldn't push the guidelines by more than 2-3x" - grinch.

 

[Scharbag]

Well, all I can do is give it a try. That or build an air conditioned and sound proofed room for my supermicro...

 

[ExcellentMong]

As a potentially easier option, could you replace the fans on the more powerful system? You mention that load is pretty low, so maybe you could afford to sacrifice some cooling performance for noise reduction.

Sent from my SM-G920I using Tapatalk

 

[Scharbag]

I have tried a bunch of different fan options with no luck :(

Because I have the 36 bay enclosure, there is a significant amount of air pressure required to push air through the drives on the front and back. No matter what I do, things overheat unless I use supermicro fans. Everything works on the lowest fan speed setting but it is still too loud for me.

Good times...

 

[snaptec]

and you won't get the 1 HE power supply fans silent anyway.
Case fans maybe...

 

[Scharbag]

The power supply fans are not so bad compared to the 7 case fans. I will try the 32 GB system. If it does not perform, I will cross that bridge at that time. Just waiting for parts to build the JBOD and then migrate everything over. Because Coral is just a little rough around the edges, and I am to busy to figure it out, I am going to stick with VMware and FreeNAS 9.10-U2. I am hoping I can just move the drives, restore my FreeNAS config and start rocking again with little effort. Already have ESXi and FreeNAS running on the 32GB machine :)

Cheers,

 

[Dice]

The power supply fans are not so bad compared to the 7 case fans.

Have you checked to where they are connected?
Attaching them to the motherboard ought to be less noisy than directly to backplanes.
I ran on 4 fans without any problems.

 

[Scharbag]

Yeah, they are plugged into my MB and the fans are set to energy saver mode.

Still too loud for me. :(

 

[Dice]

Good.
Then you are not completely deaf and still have enough human limbs working!
(I know quite exactly how much noise an old SC 847 produces in an home environment where noise cancelling has been a priority)

...have you considered doing some sort of DIY muffler box ?

 

[BarefootWoodworker]

...have you considered doing some sort of DIY muffler box ?

This was exactly what I was going to suggest. Build a box from with a door from OSB, line it with sound absorbing material, and have your intake/exhaust go around a 90* turn (though I'd probably redirect it 180*) or up and over a partition.

That will probably kill a *lot* of your sound, and since air is basically a fluid, it probably won't kill your airflow much.

 

[Dice]

Sure.
It can be done. I've one in progress. Yet, isolating vibrations is another issue that has to take place at every stage.
One day I hope I get to use my stupidly expensive and power hungry metal beast.
"soon".

 

[Scharbag]

I considered building an enclosure but I really did not have the space in my utility room. Sucks, but that is the way life goes.

I have transferred back to my old system and, thanks to VMware, the migration was pretty easy. Hardest part was transferring 36 HDDs that all had at least 4 screws in each to the Norco trays. What a tedious task...

System is up and running. I do not have a couple of parts for the JBOD case yet so I am operating without a backup pool for a small window. I just finished a scrub of the main pool and it went well. So, here is hoping that I will be able to just finish the JBOD build and plug it in to complete the migration. Norco case is SOOOOO much quieter. Not the same quality as SM, but way better for home use. I swapped the stock fans with a 120mm fan wall and all Noctua fans in the main 4220 system. The 4224 JBOD enclosure has 2 Noctua 80mm and 3 120mm Coolermasters that I had lying around so I will see how the noise is once I power it up. I expect it should be very similar to the 4220. Temperatures of the HDDs in the 4220 are right around 35C when scrubbing, so similar to the SM system. Plus, given this system is quite a bit less beefy, there are some power savings to be had.

All in all, I will miss the ridiculous overkill that the SM system offered but for what I need, the Norco solution is just fine.

Next up, I need to find a decent rack mount UPS :)

So, if there are any Canadian FreeNAS peeps out there looking for a really nice SM847 chassis, drop me a line.

Cheers,

 

[Scharbag]

So far so good with the JBOD case made from Norco hardware. Backuptank is scrubbing as we speak. HDD temperatures are:

Code:

da2: 32 C
da3: 32 C
da4: 32 C
da5: 32 C
da6: 33 C
da7: 33 C
da8: 31 C
da9: 32 C
da10: C - INTEL
da11: C - INTEL
da12: 37 C
da13: 35 C
da14: 31 C
da15: 34 C
da16: 32 C
da17: C - INTEL
da18: 32 C
da19: 32 C
da20: 32 C
da21: 32 C
da22: 35 C
da23: 34 C
da24: 35 C
da25: 34 C
da26: 34 C
da27: 35 C
da28: 35 C
da29: 36 C
da30: 36 C
da31: 34 C
da32: 35 C
da33: 35 C

Memory usage seems groovey too:

And ZFS seems to be using ARC and L2ARC effectively:

• Movies, Data, VMs shared over NFS to ESXi
  
• 8:29PM up 3:29, 2 users, load averages: 3.38, 3.17, 3.36
  
• 22.4TiB / 32.5TiB (backuptank)
  
• 27.7TiB / 43.5TiB (bigtank)
  
• 130GiB / 444GiB (fasttank)
  
• 1.46GiB / 15.9GiB (freenas-boot)
  
• 18.79GiB (MRU: 11.84GiB, MFU: 6.96GiB) / 24.00GiB
  
• Hit ratio -> 44.94% (higher is better)
  
• Prefetch -> 20.05% (higher is better)
  
• Hit MFU:MRU -> 10.52%:55.35% (higher ratio is better)
  
• Hit MRU Ghost -> 0.10% (lower is better)
  
• Hit MFU Ghost -> 8.01% (lower is better)
  

Time will tell if this bad boy works well. The case is nice and clean for the JBOD:

Tiny little Supermicro board in there even has IPMI :)

If anything goes snakey, I will let you all know.

Snoochy boochy fellow FreeNAS nerds!