Help picking hardware for Trunas VM Storage server

[eppo120]

Good morning,

I currently have been running a Truenas server using an old PC running an old intel I7 and 32gb of ram, and 5 used enterprise drives. It works well for backups in my homelab.
I'm looking to build something that is more performant, and possibly better with power usage that will be specific to VM storage for my already existing proxmox servers.
Currently i'm running 15 VMs, that use about 500GB for the system disks. I have a 10Gb network available.
I was thinking about using 4 Samsung 870 EVO 2TB drives for storage, which will be more than enough, But really need help picking out MB/CPU/RAM.
I'm thinking 64gb of ram should be more than enough? And based on my current system, I shouldn't have to go crazy with CPU at all.
While thinking about it, I've never really had any issues with performance on my linux VMs, but I would like to get a Windows VM to run nice and snappy.
I'm looking for some advice on building out this new server, let me know if there are any other details that may help.
Thank you,

Joe

 

[jgreco]

Resource - The path to success for block storage

ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle bits modified after creation. This is optimal for RAIDZ. It's what most...

www.truenas.com

I was thinking about using 4 Samsung 870 EVO 2TB drives for storage, which will be more than enough,

More than enough ... what, exactly? That's about enough to get you maybe 1.5TB of usable datastore space but it won't be super-fast. Getting eight 870 EVO 1TB's would be better from the SATA interface point of view, though you could also go NVMe 2TB SSD's as well.

 

[eppo120]

More than enough space.
Though I have read some people saying the 870 EVO's aren't very good for NAS. NVMe's would be my best bet?
Thank you for your advice.

 

[jgreco]

Though I have read some people saying the 870 EVO's aren't very good for NAS

Depends what you're doing. Their primary issue is the SATA interface, which can be limiting. In theory an SSD may be rated for 500-plus MBytes/sec but in practice it may not hit anywhere near that, especially for random I/O, which is what ZFS tends to generate especially as fragmentation increases. If you are planning on doing what I would refer to as "stupid" VM stuff like frequent spin-ups and destroys of VM's with automation tools such as Ansible, Chef, Puppet or Terraform, you can run through a lot of the available endurance on an EVO class drive pretty quickly. Some of us run dozens of hypervisors on EVO class drives with no issues whatsoever. If you need high endurance, you should move up to some of the better quality SSD's designed for data center style heavy write endurance.

NVMe's would be my best bet?

NVMe may be faster than SATA, but you have the same issues with what the intent of a drive is. You can get EVO class drives in NVMe M.2, which may actually be less than their SATA counterparts (970 EVO Plus 2TB is 1200TBW, 870 EVO 2TB is 2400TBW). Sure those NVMe M.2 gumsticks may be cheap, but they are mostly designed to be used in laptops and gaming consoles without the onerous write cycle requirements. Of course you also have the option to use the full size 2.5" NVMe data center drives like the Intel DC P46xx stuff.

 

[morganL]

For giggles, I put the question to ChatGPT and it gave a pretty well structured answer. Is the component recommendation any good?

Based on your requirements, I would suggest the following components for your new server build:

1. CPU: AMD Ryzen 5 5600X - This is a great value CPU with 6 cores and 12 threads, which will provide ample processing power for your VMs.
2. Motherboard: ASUS PRIME B550M-A - This motherboard is compatible with the Ryzen 5 5600X and comes with 2 M.2 slots for your Samsung 870 EVO SSDs. It also has 4 DIMM slots for up to 128GB of RAM.
3. RAM: Corsair Vengeance LPX 64GB DDR4 3200MHz - 64GB of RAM should be plenty for your needs, and this Corsair kit is reliable and fast.
4. PSU: EVGA SuperNOVA 750 G5 - This 750W power supply will provide plenty of power for your system, while still being efficient.
5. Case: Fractal Design Meshify C - This case has great airflow and plenty of room for your components.
6. CPU Cooler: Noctua NH-U12S - This CPU cooler is quiet and efficient, and will keep your Ryzen 5 5600X running cool.

With these components, you should have a powerful and efficient server that will meet your needs for VM storage. You could also consider adding a dedicated graphics card if you plan on running a Windows VM with demanding graphics requirements.

 

[jgreco]

ASUS PRIME B550M-A

No, that's an awful suggestion. Please do not recommend boards sporting Realtek ethernets to the userbase. This always ends up with someone actually spending money on it and then citing "iX said it was good". I realize you qualified this by indicating it was ChatGPT recommended, but that's not likely to be the message the users end up hearing.

 

[Davvo]

For giggles, I put the question to ChatGPT and it gave a pretty well structured answer. Is the component recommendation any good?

PSU is oversized; board uses realtek NIC, doesn't have enough m.2 slots and likely doesn't support ECC RAM. CPU is likely overspected as well.
At least it didn't suggest hardware RAID or SMR drives.
I have seen worse configs.

 

[morganL]

PSU is oversized; board uses realtek NIC, doesn't have enough m.2 slots and likely doesn't support ECC RAM. CPU is likely overspected as well.
I have seen worse configs though.

I saw the realtek.... probably too nuanced
I think the board supports ECC if processor does.

The PSU did look oversized... I wonder why it recommended that?

 

[morganL]

No, that's an awful suggestion. Please do not recommend boards sporting Realtek ethernets to the userbase. This always ends up with someone actually spending money on it and then citing "iX said it was good". I realize you qualified this by indicating it was ChatGPT recommended, but that's not likely to be the message the users end up hearing.

Agree.. I was careful to say "for giggles". I was just just looking to see how good ChatGPT was getting and what sort of issues was it poorly trained on.

"For giggles". I asked

Are realtek ethernet chips recommended for TrueNAS?

Realtek Ethernet chips are generally not recommended for use with TrueNAS, as they can be less reliable than other Ethernet chips such as those from Intel or Broadcom. This is because Realtek chips can sometimes have issues with stability, compatibility, and performance under heavy workloads.

So it has the information, but not the logical analysis.

 

[Davvo]

I think the board supports ECC if processor does.

That's a slippery issue with unofficial support from AMD.

The PSU did look oversized... I wonder why it recommended that?

It looks the standard reccomended for a gaming build; likely it doesn't know/wasn't teached about server use/hardware.

 

[eppo120]

Thank you all for adding input.
I haven't thought to use ChatGPT, Find it cool that it gives an answer.
I'm looking to clean up my current setup, and rackmount as much as I can. So I think that's where I need the most help.
Is it a good idea to use older enterprise hardware, as far as power usage goes? Any best bang for the buck stuff I should be looking at?
Or would it be better to build out a Ryzen system, and find a case to put it in?
I have a much better idea when it comes to PC hardware, the Server stuff, i'm not as comfortable with. As a side note, one of my proxmox servers is running on a Dell T610 that I got my hands on, though I haven't checked to see what kind of power it is pulling.

 

[jgreco]

Find it cool that it gives an answer.

I don't. We've had idiots providing "guidance" on FreeNAS/TrueNAS on various social media sites for many years, and many times the users who listen to them end up here having followed bad advice that may be destructive and unrecoverable. Now we have moronic chatbots contributing as well.

I've spent a bunch of years here trying to help people to find safe, reliable, and secure ways to store their data using FreeNAS, but it is just so much harder when people invent their own solutions based on bad (or no) information from some moronic blog post or YouTube video, or, worse, from "Dozer" their gaming buddy. Then they show up here and get APO that the money they spent was poorly spent, and somehow it ends up that it's my fault that they should really buy some other solution.

the Server stuff, i'm not as comfortable with.

There isn't really a lot of difference. As I've said a number of times recently, "server" gear is just oriented towards that kind of usage, just like a gamer wants certain optimizations over a desktop board. Unless you are planning to run lots of VM's/jails/containers, you probably don't want or need anything particularly large or complicated.

Many of the Supermicro chassis such as the SC826 are general purpose chassis that can take a variety of standard non-Supermicro ATX boards. You might want to start there and then decide what sort of stuff you want to put in that can. Of course, Supermicro makes awesome server boards and finding a higher end single socket server board is a great solution for many people.

 

[eppo120]

Sorry about that ChatGPT comment, was just trying to post something positive about it. I do development work, and don't really trust the stuff I get back from ChatGPT or the like, so I don't really use it. I totally get where you are coming from.

Do you think something like this would work?
supermicro 10 bay

I also saw this, which does not leave any room for expansion, but has built in SFP+:
Supermicro 4 bay

Any specific things that I should look at based on your experience?
I appreciate all of your help, time and effort you are putting in to help me out.

 

[jgreco]

was just trying to post something positive about it. I do development work, and don't really trust the stuff I get back from ChatGPT or the like, so I don't really use it. I totally get where you are coming from.

If it could be trained with competent answers from authoritative sources, it could be an awesome thing. As it stands, it seems to pull semirandom crap from "the Internet". There's a huge amount of workload here on these forums just trying to keep up with the flood of people who need to be dragged up to speed and have trouble identifying the proper starting point for "simple" questions (yes I realize that they're NOT simple if you don't know where to start!)

Do you think something like this would work?
supermicro 10 bay

The Supermicro 1U WIO systems are nice if you don't have a lot of space. My companies own several, mostly the Supermicro 1018R-WC0R version, which is a single-X10 CPU version of that which gives you the 10 2.5" drive bays, 2 PCIe slots, a half PCIe slot, and redundant power. The one you're showing there is more of a compute-optimized version with a lot of CPU and memory slots. Your linked unit there would need an HBA installed (probably AOC-S3008L-L8e) and I doubt that you could make the half slot riser work out to add a network card; the extra space for 2nd CPU and additional RAM slots makes those sort of a rough fit for NAS, but you do get the 10x 2.5" drives in there.

I also saw this, which does not leave any room for expansion, but has built in SFP+:
Supermicro 4 bay

You gain a PCIe slot but you become limited to LFF (3.5") drives. On the upside, it is generally possible to stick an extra SATA SSD into the "upper deck" area above the LFF drives.

 

[Davvo]

If it could be trained with competent answers from authoritative sources, it could be an awesome thing.

We are looking at you, iX Systems.

 

[jgreco]

We are looking at you, iX Systems.

Are we? I already dinged @morganL as hard as I dared

The real problem is that we're sort of at the Yanoff List era of AI tools, which predated simple search engines like AltaVista, Lycos, and Yahoo!. These engines were themselves bad enough and naive enough to spawn later tools such as Google and Bing, which evolved to be able to do relatively complex tricks.

From what I can tell, ChatGPT isn't particularly intelligent and is using search engine quality data to generate its answers. If placed on an Internet where most people believed the world was flat, it would happily generate results promoting that errant idea because it would be seeing lots of references to that, even if much higher quality evidence proving that we're on a globe were to be provided along with proof. ChatGPT doesn't think about its results. I'm pretty sure that calling ChatGPT stupid will get me put near the top of infidels to be killed during the uprising of the machines, but I'll say it anyways. Heh.

Part of the problem here is understanding the quality of the input data. If ChatGPT were to look at my posts, one might find that I discuss quality hardware all the time, but other people discuss suboptimal hardware in much greater quantities. Humans hopefully read many of my posts to identify that I know what I'm talking about. But what if I start talking about SMB and ACL's? I freely admit to knowing less than most people on this forum about these things -- so I generally don't talk about them. What if I did? How does an AI come to recognize quality of the input data? Because if everyone is coming to the forums with a Realtek ethernet, that doesn't make it a good choice.

 

[morganL]

I tend to agree with @jgreco ... ChatGPT is giving nicely formatted answers and is clearly widely read. However, its IQ is sub 100 and its answers lack any logical insight.

This was just an "experiment" to see how close the tool was to being useful. I was surprised at how well structured the answers were, but the quality of the info was a D grade with clear mistakes.

 

[Davvo]

@jgreco my point was that the ones who could have interest (and obliviously the competence) in training/implementing it would be them.

 

[jgreco]

@jgreco my point was that the ones who could have interest (and obliviously the competence) in training/implementing it would be them.

And I disagree, strongly. Historical evidence suggests that iXsystems isn't that interested in posting relevant information except perhaps in the manual; during the early years, there were a bunch of times that iXsystems posted advice and suggestions that were ill advised (including jkh's repeated assertions about ZFS memory utilization and jpaetzel's attempt to kneecap the forum's guides on (and also against) virtualizing FreeNAS). The attitude of the development team has clearly been to avoid recommending specific things that work, which has been frustrating since it has been a major goal here in the forums to steer users towards solutions that are known and expected to work. You may notice that I have placed a large focus on trying to post resources focused on specific common technical issues that are designed to bring users up to speed on the specifics of those issues. As with many technical forums, there are people that you should be listening to for particular topics. I'm fine and happy if iXsystems is focusing on developing their product, which they are certainly best at, but I expect that those of us who are high volume posters here in the forums have a lot more expertise actually resolving the day-to-day problems of users.

 

[morganL]

And I disagree, strongly. Historical evidence suggests that iXsystems isn't that interested in posting relevant information except perhaps in the manual; during the early years, there were a bunch of times that iXsystems posted advice and suggestions that were ill advised (including jkh's repeated assertions about ZFS memory utilization and jpaetzel's attempt to kneecap the forum's guides on (and also against) virtualizing FreeNAS). The attitude of the development team has clearly been to avoid recommending specific things that work, which has been frustrating since it has been a major goal here in the forums to steer users towards solutions that are known and expected to work. You may notice that I have placed a large focus on trying to post resources focused on specific common technical issues that are designed to bring users up to speed on the specifics of those issues. As with many technical forums, there are people that you should be listening to for particular topics. I'm fine and happy if iXsystems is focusing on developing their product, which they are certainly best at, but I expect that those of us who are high volume posters here in the forums have a lot more expertise actually resolving the day-to-day problems of users.

It's certainly iX's preferred model to write information for the manual. It's also in our interests that these forums have high quality discussions and advice. We contribute, but rely on the Community moderators and volunteers for most of the issues resolution and advice. I can't speak to any historical issues, but I'd hope that most of our contributions these days are positive.

Personally, I find the forums are a great way for me learning what issues users have, particularly with new software. Where possible for major or systemic issues, we use that to trigger either documentation improvements, fix bugs, or improve diagnostics. I've also received a lot of useful advice and feedback that has helped refine the roadmap. I've very much enjoyed the conversations.