Will it TrueNAS Supermicro Server

swezey

Dabbler
Joined
Feb 17, 2022
Messages
21
Newbie here... Be gentle. :smile:

So I have been asked to build a TrueNAS by my company and have never done it before. The issues is that the FIPS encrypted drives we are deploying are not supported by our installed QNAPS so here we are. Anyway, I am looking at using the following base machine to build TrueNAS on and was hoping for any insight form the community... Will it work OK? Dos and dont's? Any gotchas? Etc. Any guidance or help anyone can provide before I really get into this beast is much appreciated.

CPU: 4x Intel Xeon E7-4850 V2 12 Core 2.3Ghz ( 48 Total Cores)
RAM: 128GB DDR3 ECC REG
RAID Controller QTY 1x AOC-S3008L-L8E HBA Controller IT Mode
Caddies: 24x 3.5" Supermicro caddy
Power Supplies: 4x 1620W Power supply PWS-1K62P-1R Platinum
**8x Memory Board X10QBI-MEM1 Rev 1.01 SYS-8048B-TRFT (12 slots per board)
Supermicro 4U 24x 3.5" Drive Bays 1 Nodes
Server Chassis/ Case: SYS-8048B-TRFT
Motherboard: X10QBI Ver 1.01
Backplane: Upgrade too BPN-SAS3-846EL1 24-port 4U SAS3 12Gbps single-expander backplane, support up to 24x 3.5-inch SAS3/SATA3 HDD/SSD)
PCI-Expansions slots: 8x PCI-E 3.0 slots w/ 96 memory slots
(4x PCI-E 3.0 x16 and 4x PCI-E 3.0 x8)
*1x AOM-X10QBI-A Add-on Module support 1x VGA, Dual 10GBase-T ports X540
* Integrated IPMI 2.0 Management

Thanks for any comments, hints, suggestions, cautions, etc.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
**8x Memory Board X10QBI-MEM1 Rev 1.01 SYS-8048B-TRFT (12 slots per board)

Well that's a hell of a board. NAS generally does not need a crapton of CPU unless you are looking at running jails/VM's/containers/etc. Is there some particular reason you're going heavy? Maybe you just had it available as a used system?

CPU: 4x Intel Xeon E7-4850 V2 12 Core 2.3Ghz ( 48 Total Cores)

Those CPU's are not compatible with the X10QBI, wrong generation. You need E7 v3 or v4.

Dual 10GBase-T ports X540

10GBase-T is usually inappropriate for serious environments. Please check with your networking people to see if they would prefer SFP+. It's better, cheaper if you don't mind used, and there are great options to go to 25Gbps, 40Gbps, or even 100Gbps.

 

swezey

Dabbler
Joined
Feb 17, 2022
Messages
21
@jgreco all great points thanks!

(I don't know how to quote yet so I'll just use numbers...)

1) I heard that ZFS needs tons of RAM, the more the better. I have a bunch of DDR3 server sticks laying around so I figured why not. The Supermicro was picked up cheap and seemed like a robust platform. We are going to populate 24 slots with 12TB FIPS SED HDDs. What would be a good amount of RAM? We are figuring on RAID10 for performance and safety. Is there a better RAID level. I am new to ZFS.

2) V3, sorry my bad.

3) Our switches are Arista 10Gbe. But I am pretty sure they have a QSP 40G port. Do you know of a card that would work with the Supermicro and TrueNAS for a 40G connection?

Thanks for all the great comments - VERY helpful!

P.S. I *AM* the networking people... *LOL* 1 man show here. :cool:
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
You should check out any of the intro to ZFS pool design posts found on these forums for help with your pool design.

For commercial use, if you have 24 * 12TB disks, the basic ZFS rule of thumb of 1GB RAM per TB disk is a good idea as a starting point.

The 10 Gig Networking Primer discusses 10G cards. For 10G, the recommended cards are Chelsio T520-CR or Intel X710-DA2. For 40G, use the quad versions, i.e. Chelsio T580-CR or Intel X710-QDA2.

10G/40G networking is based on older technology, and so may be substantially cheaper used on eBay. It is being replaced by 25G/100G stuff.
 

swezey

Dabbler
Joined
Feb 17, 2022
Messages
21
@jgreco Thanks that's all great stuff! I'm sure I'll be back when I start to really get into the build!

Oh, one last thing, I'm guessing I should probably go with a DOM for TrueNAS. Any recommendations that are compatible with my chosen MOBO and what size minimum do I need for TrueNAS Core?

Thanks again!!
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
There's a SATA-DOM power header at JSD1. You can usually use the Supermicro CBL-PWEX-0625 power cable with other random brands of DOM's.

You *need* a minimum of 16GB but I would not go less than 64GB myself, just to make sure it wasn't a problem in the future.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
I have no specific knowledge of it, no reason to think it wouldn't though.
 

DCswitch

Explorer
Joined
Dec 20, 2013
Messages
58
@jgreco Do you know if would something like this work with the X10QBI?


I found a 64GB new for $50.
How's it going with your build?
I am unable to push over 450 MB/s via networking and running all the same hardware on another older system we are pulling 1900 MB/s. I cannot for the life of me figure out. How are your speeds?
 
Top