Building a "Reasonably Quiet" SSD Rack Mount Server for Office and Home use

[joeschmuck]

Backplates are generally provided with the cooling device and you pick the proper socket type, and there is an expectation that for example the LGA1200 socket will have a backplate. It's part of the socket design. So make sure that if you purchase another CPU fan, that is says it's LGA1200 compatible or you might have issues.


Good Luck.

 

[Davvo]

Also usually 155X are compatibile with 1200.

 

[bollar]

I had a nice chat with Alphacool that made me reasonably confident their water cooling solution would work and would fit in the case. With local product availability, I was able to put it all together this afternoon. The results are very good -- while Plex and the other hardware transcoding jobs are running, the TrueNAS reported CPU temperature is around 60 degrees C and the CPU-assigned chassis fans are running at 60%. Supposed TDP is 200W -- the best 2U tower cooler I could find was 125W, so this is providing quite a bit more cooling capability.

The only downside so far is PCH temperature is now 60 degrees C, probably due to restricted airflow from the radiator and all the extra stuff in the case.

As for the parts, the 2U radiator is the width of two 80mm fans, about 4mm less than 2U height and 60mm deep. There's room to add two fans in a pull arrangement if I want to try to improve the PCH temperature. It was suggested the stock Supermicro chassis fans would be able to push enough air through the radiator.

The CPU water block is the Eisblock XPX 1U. The mount is highly adjustable and wasn't difficult to install.

The system runs with a small radiator / pump combo that use a molex connector and plugs into FAN1 with a 3-pin connector.

So, much lower CPU temperature and lower fan speeds (meaning more "Reasonably Quiet")

 

[Davvo]

Would be interested in a CPU stress test temperature readings, but it looks really cool.

 

[bollar]

Would be interested in a CPU stress test temperature readings, but it looks really cool.

It's on the list for this weekend... ;)

 

[bollar]

 

[bollar]

It's going to be longer than I thought before I can bring the system down again for the stress test -- but I currently have the system busy doing nine rsync processes, along with the normal encoding and other tasks we've been discussing. Oh, and added bonus -- construction knocked out our A/C, so the room temperature is 35C.

CPU temps are about 4C higher.

Fan speeds suggest there's plenty of cooling reserve on the CPU side. The fan assigned to PCH is working much harder and is almost at max. I think I'm going to ask my better half to fashion a shroud to direct the PCH fan to that side of the board. She's good at that kind of thing. :D

Code:

CPU Temp         | 63.000     | degrees C  | ok    | 5.000     | 5.000     | na        | na        | 100.000   | 100.000 
Inlet Temp       | na         |            | na    | na        | na        | na        | na        | na        | na     
PCH Temp         | 65.000     | degrees C  | ok    | 5.000     | 5.000     | na        | na        | 90.000    | 105.000 
System Temp      | 47.000     | degrees C  | ok    | 5.000     | 5.000     | na        | na        | 85.000    | 90.000 
Peripheral Temp  | 49.000     | degrees C  | ok    | 5.000     | 5.000     | na        | na        | 85.000    | 90.000 
CPU_VRMIN Temp   | 70.000     | degrees C  | ok    | 5.000     | 5.000     | na        | na        | 100.000   | 105.000 
CPU_VRMIO Temp   | 53.000     | degrees C  | ok    | 5.000     | 5.000     | na        | na        | 100.000   | 105.000 
M2SSD1 Temp      | 62.000     | degrees C  | ok    | 5.000     | 5.000     | na        | na        | 70.000    | 75.000 
DIMMAB Temp      | 64.000     | degrees C  | ok    | 5.000     | 5.000     | na        | na        | 85.000    | 90.000 
FAN1             | 3080.000   | RPM        | ok    | 0.000     | 0.000     | na        | na        | 3220.000  | 3220.000
FAN2             | 4200.000   | RPM        | ok    | 0.000     | 0.000     | na        | na        | 7140.000  | 7140.000
FAN3             | 4200.000   | RPM        | ok    | 0.000     | 0.000     | na        | na        | 7140.000  | 7140.000
FANA             | 6020.000   | RPM        | ok    | 0.000     | 0.000     | na        | na        | 7140.000  | 7140.000

 

[bollar]

I think this project is basically done, so in case anyone else is considering a similar concept:

We are moving house and office over the next year plus and need a single server for both purposes. Our design requirement was for a reasonably quiet, portable and somewhat rugged rack-mount server that could be used in our office during the day and play DVR shows at night. Although not a requirement, we wanted Intel Quick Sync Video for hardware encoding work videos.

The universe of Xeon processors with QSV and driver support in TrueNAS-SCALE-22.12.2 is small and is basically the Intel Xeon E-2300 Series with the LGA-1200 socket. We chose the E-2388G with 8 cores / 16 threads @ 3.20GHz, which seemed the most capable.

Our preference is for Supermicro motherboards and the X12-STH-F was the logical choice (actually the only choice, since the other LGA-1200 boards were for specialized purposes). The microATX form factor was a bonus.

We were looking at 2U SSD chassis and had lots of choices. SAS3 was the only requirement. Current generation chassis supporting NVMe were quite expensive and we found previous generation coming off lease at a reasonable discount. We chose the SuperMicro SuperChassis 216BE1C-R920LPB.

We used the standard Supermicro cards for HBA & 10GBe.

Assembly was straightforward, but we found the CPU temperature hovered at 80C. Although we were advised that temperature was "normal" for QSV-enabled CPUs, it did make the fans very loud. Not finding an air cooler with dramatically better performance than the Arctic Freezer 11 LP we originally selected, we decided to try a custom liquid cooling solution for 2U servers. Though expensive, it wound up working very well and easily keeps the CPU at 60C in a 37C room.

We heard the advice about not using cheap SSDs, but with local and offsite backups, decided to take some risk in exchange for convenience and structured the main pool as 3 VDEVs of RAIDZ2 with Samsung 870 QVO 8TiB drives. We added a metadata & < 64K block VDEV with 3-way mirrored and striped Samsung 850 Pro 2TiB SSDs. The rationale being to limit writes to the cheap drives.

iX Apps are on their own 2-way mirrored Samsung 980 Pro 2TiB NVMe. There's also an Intel Octane M.2 that we haven't decided what to do with yet. It's working as a transcoding scratch directory for now.

Usable Capacity is 124 TiB.

The server fits in a large suitcase and weighs just under fifty pounds including the drives.

Pricing, for what it's worth. The CPU was more expensive than I had expected, and ECC RAM was harder to find than I expected.

SSD pricing was very volatile. I wound up buying over several days from eight vendors.

 

[joeschmuck]

That is one very expensive CPU cooler.

This project seems more like a workstation with lots of good storage than a NAS device. But I understand those were your needs.

I'm glad you are happy with your project.