Regarding hardware for first TrueNAS build.

fobo

Cadet
Joined
Sep 13, 2021
Messages
4
Good evening good people!

First of all I would like to say thanks to the contributors of this forum, the knowledge base here is immense, really.

English is not my first language so I apologize for any spelling/grammar mistakes.

Having searched and read the TrueNAS forums and reddit, every evening recently, and having read the "Hardware Recommendations Guide" from back to back, I am still uncertain about what mainboard and CPU to choose in the end. I am thankful for any input you might have.

I started out thinking (like many others) that I was going to use my old i7 4770k and z87 motherboard and 32GB DDR3 RAM for this build. That was until I took a deep dive into these forums and got lectured in ECC, ZFS and the quality of server/enterprise components vs. consumer parts etc. I have built desktop computers for 20 years, but never something like this.

I will try to make this as small a wall of text I can!

My usage:
- Plex, with mainly direct play/DLNA use for streaming to home theater room, very little transcoding, if any. About 600, 4K, 40-80GB each, movie files.
- Main storage for my desktop (connected with 10gigabit NIC), including a full steam library folder. Running steam client on 1 PC, with SMB share to library folder.
- Backup "cloud like" file server for me and 2-3 family members.
- Want to run it for atleast 5 years.


Allready bought/story so far:

HDD
: 8x 14TB enterprise drives, Toshiba MG07ACA14TE. (Thinking Raidz2) I understand now that this might have been very a bad idea, having read about resilver times for large drives etc. 112TB Total.
NIC: 2x Genuine Intel X520 10gbe NICs, one for NAS and one for main desktop. Putting 40mm fans on these.
CHASSIS/COOLING: Fractal Define R6, with 8 hdd sleds, and 5 monster 120mm chassis fans. Noctua LGA 115x/2011 CPU cooler.
PSU: Seasonic 750W PSU.
Boot drives: 120GB Kingston SATA SSD, mirrored.

What now?

I see users adding HBA cards, even though they have more than enough SATA ports on their motherboard, like the LS cards. Is there a practical reason for this?

I was almost set on a X11SSM-F + HBA build. But then I read iXsystems and these forums often recommend 1GB of ECC RAM for each 1TB of storage, but the most recommended mainboards here on the forums are 64GB max, like the Supermicro X11-F cards. The LGA115x i3-9100 is often recommended instead of a Xeon CPU because of the ECC support, this only support 64GB of RAM. This is half the amount recommended compared to the the 112TB pool I am building. How would this affect the performance of the NAS if I go this route, would it be a viable option?

The second hand enterprise server market here in Norway is awful. So if I am going that route, my options are ebay/amazon etc.

That considered, I really like the thought of going with the X9 series and the LG2011 socket, and DD3 RAM. Something like a used X9SRi-F that the user ChrisRJ has suggested in lots of threads like this one on this very forum. And just going 128GB+ RAM as it is dirt cheap compared to DDR4. This board also have enough SATA ports onboard, so HBA is not needed. How would this route compare to a newer x11/x12 build, what do you lose? And If I were to go the X9 route, what particular Xeon CPU would be the best match, performance wise, with my usage goals?

I have no particular budget, but trying to be sensible.

Thanks again, have a great day! :)
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
(Thinking Raidz2

Indeed, RaidZ is what will serve you the best. To serve a few large file at a time is often better achieved with RaidZ than with mirrors. Mirrors offer more IOPS but that is needed for VM, block storage and high number of concurrent access. Considering your situation, RaidZ is the way to go.

At 8 drives, you are well within the safety margin for RaidZ2. Do not waste time whenever one of your disk fails, be sure to monitor them with smart tests and have a spare on site. Still, no single server, TrueNAS or other, can be more than a single point of failure. You need to plan for backups. See my signature for a complete plan.

1GB of ECC RAM for each 1TB of storage

Do not worry yourself about that rule of thumb. It is really not a hard requirement. Considering how a small part of your total volume will be access at any single time, you will not be in trouble. TrueNAS will never have too much ram, but no panic here.
 

fobo

Cadet
Joined
Sep 13, 2021
Messages
4
Indeed, RaidZ is what will serve you the best. To serve a few large file at a time is often better achieved with RaidZ than with mirrors. Mirrors offer more IOPS but that is needed for VM, block storage and high number of concurrent access. Considering your situation, RaidZ is the way to go.

At 8 drives, you are well within the safety margin for RaidZ2. Do not waste time whenever one of your disk fails, be sure to monitor them with smart tests and have a spare on site. Still, no single server, TrueNAS or other, can be more than a single point of failure. You need to plan for backups. See my signature for a complete plan.



Do not worry yourself about that rule of thumb. It is really not a hard requirement. Considering how a small part of your total volume will be access at any single time, you will not be in trouble. TrueNAS will never have too much ram, but no panic here.
Thank you for your swift reply! Any thought regarding my inquiry about CPU/MB? Have a great evening! :)
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
Any thought regarding my inquiry about CPU/MB

All of my 3 TrueNAS servers have been bought new.... I never mix-n-matched components like fans, power supplies, etc.
 

ChrisRJ

Wizard
Joined
Oct 23, 2020
Messages
1,909
That considered, I really like the thought of going with the X9 series and the LG2011 socket, and DD3 RAM. Something like a used X9SRi-F that the user ChrisRJ has suggested in lots of threads like this one on this very forum. And just going 128GB+ RAM as it is dirt cheap compared to DDR4. This board also have enough SATA ports onboard, so HBA is not needed. How would this route compare to a newer x11/x12 build, what do you lose? And If I were to go the X9 route, what particular Xeon CPU would be the best match, performance wise, with my usage goals?
Glad, if I could "help". I am running a 4 core E5-1620 at 3.6 GHz (4 cores, 8 threads) and it is pretty idle. If you want Plex to do online transcoding this might be a different story though. For my 8*16 TB in RAIDZ2 I have 64 GB and, as @Heracles wrote, the "1 GB RAM per 1 TB storage" is more geared towards lower amounts, so that people don't think that 200 TB storage and 8 GB RAM might be a suitable combination. The X9 boards require Java for the virtual KVM, which is far from ideal. But that is an individual thing to decide.
 

ChrisRJ

Wizard
Joined
Oct 23, 2020
Messages
1,909
CHASSIS/COOLING: Fractal Define R6, with 8 hdd sleds, and 5 monster 120mm chassis fans. Noctua LGA 115x/2011 CPU cooler.
Forgot: I have this and the sleds for the disks make cooling pretty challenging. They cover the disks from both sides and the bottom. So cooling air can only work on the top of the drives. And the space between a drive's top side and the bottom side of the sled above it is about 2 mm (or less). Depending on the disk temperature you want, this may require some beefy (and loud) fans.
 

fobo

Cadet
Joined
Sep 13, 2021
Messages
4
Glad, if I could "help". I am running a 4 core E5-1620 at 3.6 GHz (4 cores, 8 threads) and it is pretty idle. If you want Plex to do online transcoding this might be a different story though. For my 8*16 TB in RAIDZ2 I have 64 GB and, as @Heracles wrote, the "1 GB RAM per 1 TB storage" is more geared towards lower amounts, so that people don't think that 200 TB storage and 8 GB RAM might be a suitable combination. The X9 boards require Java for the virtual KVM, which is far from ideal. But that is an individual thing to decide.
In the search for a decent X9 board, I came across a small second hand tower server for sale here in Norway with these speccs: X11SSH-LN4F, 64GB DDR4 ECC RAM, 3.60Ghz Intel Xeon E3-1275 v5, Samsung PM961 256GB NVMe SSD, 500W gold PSU, and ITX chassis. Seller was asking 500$ and I just went for it. I could not find the parts cheaper on any site, when including import taxes and shipping. MB has 8 SATA, 1 M.2 so it was a good fit for the 8 drives i have bought. The CPU is "only" 4 core 8 threads but I hope it will suffice. Thank you for your input!
 

fobo

Cadet
Joined
Sep 13, 2021
Messages
4
Forgot: I have this and the sleds for the disks make cooling pretty challenging. They cover the disks from both sides and the bottom. So cooling air can only work on the top of the drives. And the space between a drive's top side and the bottom side of the sled above it is about 2 mm (or less). Depending on the disk temperature you want, this may require some beefy (and loud) fans.
Thanks for your input, after having seen pictures of a fully populated hdd rack on this tower, I agree that it looks toasty. This post made me think. I started searching through tons of different tower options, including all the offerings from Fractal, both new and old. During this search, I came across the Fractal Define S (first version, not S2), that is based on the R5 interior, but lacking all the 3,5"/5.25" cages. It has 3 big 140mm fans in the front, and a magnetic dust filter. I noticed it still has all the screw holes and mounts for the hdd cages in the bottom, to fit the hdd cages from the old Define R4/R5.

So I found a used Define S cheap, and I am going to try to flip the entire hdd cage with 8x 3,5" sleds from a used R4, and mount it sideways in the Define S, so the three 140mm fans blow straight through the front of the cages, like an air tunnel. With almost 10mm between the drives, and no steel to stop the airflow, I am optimistic.

Waiting for it to arrive, will post a picture of this Frankenstein if it succeeds, hehe.
 
Top