64gb server recommendations

[Malakym]

I'm currently on a Supermicro X10SL7-F with a v3 E3, limited to 32gb of RAM (so are all the v4 E3s).

Struggling a bit with 32gb and could do with upgrading but cannot go any further than 32gb with the current motherboard/CPU setup.

Any suggestions? I can find a few v5 processors cheap (~£25) but all the motherboards are expensive even when used (£220+).

I use a h200+expander so only need the PCIe slots for that.

My supermicro chassis/board uses a normal motherboard IO plate so flexible in that respect.

 

[TXAG26]

I have the same setup with 32GB ram and if you’re just running TrueNAS, it should run well with that amount. Mine is used as an offline backup destination and it handles a 10GbE connection just fine. However, if you’re running jails and other stuff on the machine, I could see those other uses needing more ram. I think the general consensus now is 32GB of ram can basically run any size pool, with a slight drop in performance as extra ram is just used for cache/swap for the most part. E5-2600 v4 CPU’s have gotten very reasonably priced, but the motherboards remain stubbornly expensive. If you can find a Supermicro X10 E5-2600 v4 board for around $150-$175, that’s a pretty good deal from what I’m seeing. Most of those boards have 8 ram slots and can cheaply take you to 64/128/256GB if that’s what you’re looking for.

 

[Malakym]

Not worried about RAM for ZFS pools, can run with extremely small amounts with no issue.

FYI the board needs to be matx. Most of the cheap E5s tend to be dual processor sadly. I had noticed how cheap the processors are too, just struggling to find a matching mobo.

 

[TXAG26]

Take a look at the Supermicro X10SRM-F. With 32GB RDIMMS you can easily get to 128GB.

X10SRM-F | Motherboards | Products | Super Micro Computer, Inc.

X10S, X10SR, X10SRM, X10SRM-F

www.supermicro.com

 

[TXAG26]

There is also a X10SRM-TF version that has 2x Intel x500 10GbE ports (the X10SRM-F has 2x Intel i350 1GbE ports). Both boards come with Supermicro’s standard IPMI (BMC).

X10SRM-TF | Motherboards | Products | Super Micro Computer, Inc.

X10S, X10SR, X10SRM, X10SRM-TF

www.supermicro.com

 

[ChrisRJ]

Not worried about RAM for ZFS pools, can run with extremely small amounts with no issue.

FYI the board needs to be matx. Most of the cheap E5s tend to be dual processor sadly. I had noticed how cheap the processors are too, just struggling to find a matching mobo.

I am really confused by this. Can you perhaps rephrase or go into some more detail? Or someone else, of course?

 

[jgreco]

I am really confused by this. Can you perhaps rephrase or go into some more detail? Or someone else, of course?

Yeah, it's weird. You shouldn't try to starve ZFS of RAM, it may eventually fail to be able to import its pool; it's suspected that many of the early failures with FreeNAS involved people trying to go too small on RAM. As for CPU's, the X10SR* boards can take either E5-16xx v3 or v4 CPU's, or E5-26xx v3 or v4 CPU's. Huge amount of flexibility.

 

[Malakym]

I am really confused by this. Can you perhaps rephrase or go into some more detail? Or someone else, of course?

The suggested high ram requirements of ZFS isn't nearly as bad as some people suggest. I need more for jails/vms.

As for CPU's, the X10SR* boards can take either E5-16xx v3 or v4 CPU's, or E5-26xx v3 or v4 CPU's. Huge amount of flexibility.

Will have a look, only the SRM-F* seems to be matx so far and pretty expensive even used. Doesn't need to be supermicro mind you.

 

[jgreco]

There really aren't a lot of manufacturers of server boards that are in standard form factors.

 

[TXAG26]

Could be cheaper to upgrade your chassis and go with a standard sized ATX board. What chassis are you using?

 

[Malakym]

My problem is the depth of the chassis, I use a short supermicro that fits in a "deep" network cabinet.

Looking at the price of some of the options at the moment, I might be better off with a second server in its own chassis... even if I was limited to 32gb again.

 

[TXAG26]

How many drives and what physical format are they in your current chassis?