Building my new server....trouble selecting parts

KRDucky

Dabbler
Joined
Apr 16, 2019
Messages
15
Greetings all,

My current server is a Dell PowerEdge T320 with a 6 core, 12 thread Xeon Ivybridge E5. I have 64GB of DDR3 ECC. I also have 8x4TB WD Red CMR drives. As well as a Samsung 970 Evo NVMe SSD for caching. My current server OS is Ubuntu Server 20.04 with my storage pool being ZFS in either a Nested RAIDz, Mirrored RAIDz or Striped RAIDz. I cant really remember the command I typed to make the array. I do know it is 2 RAIDz sets in the same pool.

Here is a image of my pool layout (used Cockpit's ZFS plugin to capture this image):

ZFSPool.png



I do not want to spend more than 1200 on parts. I mainly use this for serving media to my TV and when Im on the road. I run Plex and Jellyfin (because sometimes one works when the other doesnt). For transcoding I have a Nvidia Quadro P2200. All my services run in Docker behind a Linuxserver.io SWAG reverse proxy.
I am VERY interested in moving to TrueNAS SCALE as it is seemingly the best of both worlds (FreeNAS + Linux). My server is connected to a APC Back-UPS Pro 1300 S via the USB connector.

My current server is LOUD and HOT. It doubles as a heater for my apartment in the winter and overloads my AC in the summer. It is easily the loudest thing in my apartment. I can still hear it through Active Noise cancellation headphones.


Thus, I want to replace it with a quieter, cooler server.

I have found some Supermicro Denverton Atom embedded boards that seem cool since I wouldn't need my PERC H310 card anymore as they have onboard mini-SAS HD ports. I have also seen the embedded Epyc boards. If I wasnt using ZFS, maybe ECC wouldn't be that big of a thing to me but it is. ( having 32TB raw storage helps too).

Cases are also a thing I have difficulty with as I enjoy Hot-swap bays but can live without them if necessary. I have tried to find the top-to-bottom 5.25 bay cases like the Antec 1200 and Antec 900 but they are not sold anywhere I can find. I had thought to use the Silverstone or Icydock 3x5.25 to 4x3.5 bay kits to get my hotswap fix.

Supermicro has some workstation cases that would do it but those alone are easily ~$500.

I am personally not a fan of putting a rack in my small apartment. Not just from the space constraints but also the noise. I would have to seriously mod one to reduce the noise.

As it is, I have kind of settled on the Fractal Design Node 804. I will be moving my 8x WD Reds over.
I will need to find ( unless I am missing something here):

  • ECC RAM
  • Mobo/CPU
  • PSU

I do have a router with a single SFP+ port on it ( Ubiquiti EdgeMax SFP+)

I would appreciate any assistance it picking the parts I would need for my build.

What I currently have that I would be using in the new TrueNAS SCALE build:

  • Nvidia Quadro P2200 (Transcoding card)
  • PERC H310 (IT Mode) - If the motherboard doesnt have SAS ports for SATA breakout cables
  • Samsung 970 Evo NVMe (ZIL or L2ARC cache, may add another)
  • 8x 4TB WD RED HDDs (SATA)
  • Crucial MX500 SSD (OS Drive)
 

KRDucky

Dabbler
Joined
Apr 16, 2019
Messages
15
hmm. I have an ADATA SX8200 not an Samsung 970 Evo.... could have sworn I used a Samsung NVMe drive.
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,107
Neither SX8200 or 970 Evo are suitable for a SLOG, which does not appear useful for your use case anyway. They could be L2ARC, if useful.

A loaded Node 804 is not silent, as the HDD noise directly gets out from the top, but is otherwise a fine case—and probably quieter than a typical rackmount server.
Possible MB choices could be:
a/ C3000-based board (e.g. Supermicro A2SDi). C3758 if you want 10 GbE (A2SDi-H-TF) and/or a full 12 SATA ports.
b/ MicroATX ASRockRack D1541D4U-2T8R or FlexATX members of the X10SDV family with an onboard LSI controller.
c/ MicroATX server board with a Core i3-8100/9100 processor (e.g X10SCH-F), and your H310 HBA.
Option c/ is cheapest, but requires UDIMM, which is more expensive than RDIMM for a/ or b/. It is potentially upgradable… but likely will never be, and is already an obsolete socket. Look out for second-hand opportunities for option b/ or even a/ (less likely).

If this is merely serving media to your TV (over 1 GbE I suppose), 10 GbE is overkill. It would be useful when serving files to a desktop PC which has 10 GbE, in which case you need a switch with at least two ports such as ASUS UG-X2008 (10 Base-T) or Microtik CRS305-1-4S+IN (SFP+).
 

c77dk

Patron
Joined
Nov 27, 2019
Messages
467
Have to agree on the 804 sending the noise right up - maybe take a look at the Define R7 - got the R6 under my table and combined with 3 Noctua 120mm fans in front (3x120mm is quieter and gives better airflow than 2x140mm) and a 140mm in the back it doesn't say much
 

ChrisRJ

Wizard
Joined
Oct 23, 2020
Messages
1,903
When I built my current FreeNAS box the noise was the main reason not go for a rack-mount case. Instead I chose a Fractal Define R6, which is big enough for my 8 drives. It turns out, though, that it is not so silent, unless I want the drives to run at around 40 degrees Celsius. So I installed beefier fans (2x140 mm) and the noise is now roughly the same as with my 1U Supermicro boxes. My guess is that finding a case for 8+ drives that allows for relatively quiet cooling might be harder than modifying a rack-mount case to run not so loud.
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,107
The point is, the best way to remain silent is to have as few drives as possible. Which may not be convenient now that large HDDs are hoarded by Chia farmers.
If you do go for an ATX case I would recommend the Nanoxia Deep Silence 8 Pro over the Fractal Define series. The Nanoxia is obviously a cheaper knock-off of the Define, but for hosting drives it is actually more convenient. "Storage layout" with the Define requires repositioning a panel and screwing each drive to its carrier (8 HDD * 4 screws :frown: ). The Nanoxia ships ready to install drives in its tool-less caddies.
 

KRDucky

Dabbler
Joined
Apr 16, 2019
Messages
15
The point is, the best way to remain silent is to have as few drives as possible. Which may not be convenient now that large HDDs are hoarded by Chia farmers.
If you do go for an ATX case I would recommend the Nanoxia Deep Silence 8 Pro over the Fractal Define series. The Nanoxia is obviously a cheaper knock-off of the Define, but for hosting drives it is actually more convenient. "Storage layout" with the Define requires repositioning a panel and screwing each drive to its carrier (8 HDD * 4 screws :frown: ). The Nanoxia ships ready to install drives in its tool-less caddies.
When I built my current FreeNAS box the noise was the main reason not go for a rack-mount case. Instead I chose a Fractal Define R6, which is big enough for my 8 drives. It turns out, though, that it is not so silent, unless I want the drives to run at around 40 degrees Celsius. So I installed beefier fans (2x140 mm) and the noise is now roughly the same as with my 1U Supermicro boxes. My guess is that finding a case for 8+ drives that allows for relatively quiet cooling might be harder than modifying a rack-mount case to run not so loud.

Ive been considering the Phanteks Enthoo Pro 2 (what my new desktop is in), the Phanteks P600S and the Fractal Design 7XL.

In terms of noise, while yes the 5400RPM drives are going to be loud-ish.....I would argue that the vast majority of the noise my current server makes is from the SINGLE 6000RPM chassis fan at the rear that cools the CPU, RAM and HDDs.

The HDDs probably make the most sound from vibration. IIRC, both Phanteks and Fractal have HDD dampening buffers so that should reduce the vibration noise. As far as cooling goes, Both the Enthoo Pro 2 and Define 7XL support a crapload of fans.


Neither SX8200 or 970 Evo are suitable for a SLOG, which does not appear useful for your use case anyway. They could be L2ARC, if useful.

A loaded Node 804 is not silent, as the HDD noise directly gets out from the top, but is otherwise a fine case—and probably quieter than a typical rackmount server.
Possible MB choices could be:
a/ C3000-based board (e.g. Supermicro A2SDi). C3758 if you want 10 GbE (A2SDi-H-TF) and/or a full 12 SATA ports.
b/ MicroATX ASRockRack D1541D4U-2T8R or FlexATX members of the X10SDV family with an onboard LSI controller.
c/ MicroATX server board with a Core i3-8100/9100 processor (e.g X10SCH-F), and your H310 HBA.
Option c/ is cheapest, but requires UDIMM, which is more expensive than RDIMM for a/ or b/. It is potentially upgradable… but likely will never be, and is already an obsolete socket. Look out for second-hand opportunities for option b/ or even a/ (less likely).

If this is merely serving media to your TV (over 1 GbE I suppose), 10 GbE is overkill. It would be useful when serving files to a desktop PC which has 10 GbE, in which case you need a switch with at least two ports such as ASUS UG-X2008 (10 Base-T) or Microtik CRS305-1-4S+IN (SFP+).
I think I have at most 10 machines that connect to my server (hopefully not at the same time). A couple of friends are connected to my Plex and my folks get to watch all their VHS movies I backed up for them.
Will a Denverton C3000 Atom be sufficient given the workhorse is the P2200? Or should I opt for the Epyc SoC? Does the Core i3 chip even support ECC? One would think that ECC would be kind of important.

I had also considered getting this CPU:
Intel Xeon E-2146G Coffee Lake 3.5 GHz LGA 1151 80W

Would that work? And do you have any mobo selections for that?
 
Last edited:

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,107
There are no 5400rmp NAS drives. But WD's marketing department sells some 7200rpm drives as "5400rpm-class", whatever that means.

Up to the 9th generation, Core i3s support ECC. A Core i3-9100F is a Xeon E-2224 in all but name and price. For more cores with ECC you have to pay for the Xeon E brand. Motherboards for Xeon E-2xxx are the same as for using Core i3-8/9xxx with ECC: C242 or C246 chipset, such as X11SCH-F.
i3-10xxx no longer support ECC.

If transcoding is uploaded to a dGPU, I suppose that you don't need a iGPU (small saving) and that an Atom C300 suffices to shuffle data around, but I have no practical experience with that.

EPYC looks like overkill, and embedded EPYC would likely raise cooling issues in consumer cases. Atom C3000 or Xeon D-1500 are easy to cool with a taped-on 60mm fan.

If you go for ATX size, the possibilities expend considerably. I suppose that 10 GbE could serve an additional criteria. Still, to save money look at second-hand Xeon hardware with RDIMM.
 

KRDucky

Dabbler
Joined
Apr 16, 2019
Messages
15
There are no 5400rmp NAS drives. But WD's marketing department sells some 7200rpm drives as "5400rpm-class", whatever that means.

Up to the 9th generation, Core i3s support ECC. A Core i3-9100F is a Xeon E-2224 in all but name and price. For more cores with ECC you have to pay for the Xeon E brand. Motherboards for Xeon E-2xxx are the same as for using Core i3-8/9xxx with ECC: C242 or C246 chipset, such as X11SCH-F.
i3-10xxx no longer support ECC.

If transcoding is uploaded to a dGPU, I suppose that you don't need a iGPU (small saving) and that an Atom C300 suffices to shuffle data around, but I have no practical experience with that.

EPYC looks like overkill, and embedded EPYC would likely raise cooling issues in consumer cases. Atom C3000 or Xeon D-1500 are easy to cool with a taped-on 60mm fan.

If you go for ATX size, the possibilities expend considerably. I suppose that 10 GbE could serve an additional criteria. Still, to save money look at second-hand Xeon hardware with RDIMM.

According to Intel, the 10th and 11th gen CPUs do support ECC. You just have to select the right model.

https://ark.intel.com/content/www/us/en/ark/search/featurefilter.html?productType=873&0_ECCMemory=True&1_Filter-Family=122139&0_HyperThreading=True
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,107
Ah, "E" parts for "embedded". Thanks, I've learned something! But these are not the regular i3-10100 and retail availability could be an issue…

Your list should make a workable system. The X11SCA-F is a workstation board; it does have a BMC and can certainly make a decent server, but unless you plan to reuse it as a desktop PC later on, you're wasting money on unnecessary audio codec, video ports (especially with a E-2236) and PCI slot and not really making use of ATX size. X11SCH/L/M-F are the pure server boards for this generation.
If noise is a concern, the stock Intel CPU cooler may not be the best.
 

KRDucky

Dabbler
Joined
Apr 16, 2019
Messages
15
Ah, "E" parts for "embedded". Thanks, I've learned something! But these are not the regular i3-10100 and retail availability could be an issue…

Your list should make a workable system. The X11SCA-F is a workstation board; it does have a BMC and can certainly make a decent server, but unless you plan to reuse it as a desktop PC later on, you're wasting money on unnecessary audio codec, video ports (especially with a E-2236) and PCI slot and not really making use of ATX size. X11SCH/L/M-F are the pure server boards for this generation.
If noise is a concern, the stock Intel CPU cooler may not be the best.
can you suggest a better board then? I plan to use Noctua for my fans and CPU cooler. Something like the NH-D15 Chromax or NH-U14.

I have modified my CPU selection to the E-2226G: E-2226G

Im looking at 7x Noctua A14 PWM Industrial 2000 case fans to handle case cooling.
 
Last edited:

ChrisRJ

Wizard
Joined
Oct 23, 2020
Messages
1,903
Would something with DDR3 ECC RDIMMs be an option? You could safe a lot of money there, depending on how much memory you want.
 

KRDucky

Dabbler
Joined
Apr 16, 2019
Messages
15
Would something with DDR3 ECC RDIMMs be an option? You could safe a lot of money there, depending on how much memory you want.
Well, that would work if I was willing to drop down to Haswell. But Haswell isnt exactly known for efficiency and running cooler. I recently upgraded from a i7-4770k to a Ryzen 9 5900X.

IF I could use Ryzen 3900X for example with ECC RAM....that would be a no-brainer. Especially since I plan to use TrueNAS SCALE.
The main issue I find with a lot of these motherboards is given the cost of the SoC boards, why is there only 4-6 SATA ports and no SAS Mini HD ports or say 4 SATA and an OCULiNK?

I want to go efficiency, cool and quiet. hence the Full ATX chassis and crapload of Noctua fans.
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,107
can you suggest a better board then? I plan to use Noctua for my fans and CPU cooler. Something like the NH-D15 Chromax or NH-U14.
Such big coolers are probably overkill, but won't hurt.
The corresponding server boards with 8 SATA would be X11SSH-F (2*M.2 for, say, boot and possible L2ARC; but slightly more expensive and apparently hard to find now, tough that's probably temporary) or X11SSH-LN4F (1*M.2 and 4*LAN, which is probably not of much use unless you can team them, but available and less expensive than X11SCA).

Well, that would work if I was willing to drop down to Haswell. But Haswell isnt exactly known for efficiency and running cooler. I recently upgraded from a i7-4770k to a Ryzen 9 5900X.
A NAS CPU is close to idle most of the time…
Going for "real" Xeons (above E3/E-2xxx/W-1xxx which are Core CPU with ECC) gives access to more PCIe slots and to RDIMM, which is cheaper than UDIMM and allows for more total RAM (always a no-brainer upgrade when possible).
Going for second-hand old Xeons and DDR3 allows for really big savings. If you're ready to put a NH-D15 or NH-U14 on a LGA2011 Xeon, that is an option to consider.

If I could use Ryzen 3900X for example with ECC RAM....that would be a no-brainer. Especially since I plan to use TrueNAS SCALE.
The main issue I find with a lot of these motherboards is given the cost of the SoC boards, why is there only 4-6 SATA ports and no SAS Mini HD ports or say 4 SATA and an OCULiNK?
You should have said so from the beginning :wink:
SCALE is still in beta, so not yet advisable for important data, and Ryzen support on CORE is far less advanced than support for Intel but you would not be alone in this forum to run TrueNAS on Ryzen. The manufacturer to look at is then AsRockRack rather than Supermicro.
X570D4U: 8 SATA, PCIe x16/x8 + x8/x0
X570D4U-2L2T: same with onboard 10 GbE (review)
 

KRDucky

Dabbler
Joined
Apr 16, 2019
Messages
15
Such big coolers are probably overkill, but won't hurt.
The corresponding server boards with 8 SATA would be X11SSH-F (2*M.2 for, say, boot and possible L2ARC; but slightly more expensive and apparently hard to find now, tough that's probably temporary) or X11SSH-LN4F (1*M.2 and 4*LAN, which is probably not of much use unless you can team them, but available and less expensive than X11SCA).


A NAS CPU is close to idle most of the time…
Going for "real" Xeons (above E3/E-2xxx/W-1xxx which are Core CPU with ECC) gives access to more PCIe slots and to RDIMM, which is cheaper than UDIMM and allows for more total RAM (always a no-brainer upgrade when possible).
Going for second-hand old Xeons and DDR3 allows for really big savings. If you're ready to put a NH-D15 or NH-U14 on a LGA2011 Xeon, that is an option to consider.


You should have said so from the beginning :wink:
SCALE is still in beta, so not yet advisable for important data, and Ryzen support on CORE is far less advanced than support for Intel but you would not be alone in this forum to run TrueNAS on Ryzen. The manufacturer to look at is then AsRockRack rather than Supermicro.
X570D4U: 8 SATA, PCIe x16/x8 + x8/x0
X570D4U-2L2T: same with onboard 10 GbE (review)

An old friend of mine just offered me his old Threadripper 1950X and the motherboard for $350 shipped. I realize that is probably overkill for a NAS/Media server but......I definitely would not be limited in what I could do. Should I jump on it?
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,107
"À cheval donné, on ne regarde pas les dents." (On a given horse, don't check the teeth. — French saying)

Overkill, yes. Possibly not that efficient for sitting mostly idle while serving files.
But certainly an opportunity for a server, or for a workstation and then recycling your current desktop into a server.
 

KRDucky

Dabbler
Joined
Apr 16, 2019
Messages
15
Currently, I am focusing on buying the case and anything I can other than the motherboard and CPU since picking one out is being a chore.

The two cases I am trying to decide between are the Phanteks Enthoo Pro 2 and the Fractal Design 7XL.....
What are ya'll's pros and cons for either?
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
On a given horse, don't check the teeth. — French saying
Don't look a gift-horse in the mouth - English saying (that of course sounds better as an "old" English phrase)
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,107
Don't look a gift-horse in the mouth - English saying (that of course sounds better as an "old" English phrase)
"Old" as in the times when the kings of France and England shared a bed? (Philip August and Richard "Lionheart")
Thanks for bringing up the English saying. It sound better than my attempt at a translation.
 
Top