Xeon server build, looking for recommendations on parts please

Deeda

Explorer
Joined
Feb 16, 2021
Messages
65
Thanks, though I think you misunderstood. I've already done the flash using sas3flash via FreeDOS.

My point was that the custom TrueNAS firmware version doesn't include the BIOS and UEFI BSD updates. If you were updating from a really old version, I'm assuming you need update those as well, but have to grab them from the 16.00.10.00 package.
 

neofusion

Contributor
Joined
Apr 2, 2022
Messages
159
Thanks, though I think you misunderstood. I've already done the flash using sas3flash via FreeDOS.

My point was that the custom TrueNAS firmware version doesn't include the BIOS and UEFI BSD updates. If you were updating from a really old version, I'm assuming you need update those as well, but have to grab them from the 16.00.10.00 package.
Yes, you're fine with using the BIOS/UEFI bits from the most recent normal package.
 

Deeda

Explorer
Joined
Feb 16, 2021
Messages
65
I have seen connectors with data and power combined, but they have always had a place to insert a separate power cable right into the back of the bigger combined contact.

Like so:
View attachment 56504
Do yours have something similar? It's difficult to see in the picture you provided.
You were indeed correct, the cables do have the SATA power connector in the location shown in your photo... but... they don't work anyway :eek:

Was spending some more time on the build last night... I used the SAS cables that came with the motherboard to connect SAS drives to the onboard controller, and I used these 10Gtek cables to connect other SAS drives to the PCIe SAS controller: https://www.amazon.com.au/gp/product/B08B1NWP67/ref=ppx_yo_dt_b_asin_image_o01_s00?ie=UTF8&psc=1

The drives connected to the 10Gtek cables showed up fine, but not the ones connected to the cables that came with the motherboard (Foxconn I believe). I then connected the 10Gtek cables to the onboard SAS controller and the drives showed up fine. So that narrows it down to a cable issue I guess.

Not sure why the ones included with the motherboard don't work. I can buy more 10Gtek ones I guess, but then need to somehow work out how I am going to find another 8 x molex connectors.

EDIT: now wondering if this might be the cause: https://www.youtube.com/watch?v=fnISM_LMuss
 
Last edited:

Lemming

Cadet
Joined
May 5, 2014
Messages
7
Ok talking to a friend who is an expert on servers. They said I'll need at least 3000w for the amount of hard drives I am putting in. Looking at one of these https://usa-m.banggood.com/3450W-Mi...05.html?akmClientCountry=AU&a=1656043150.8113

I'm sure a number of you here have used the same in your builds.
Sorry but saw this and just had to reply.

My current system is a Dual Xeon Platinum 8253 (125W TDP), 256GB RAM, 2 LSI 3008 HBA's, Dual port 10GBe card, Sun F80 Flash Accelerator, 12 6TB SAS drives, 12 4TB SAS drives, 4 1TB SATA SSD's, 2 x 512GB M.2 NVME SSD's in a Supermicro CSE847 36Bay 4u case.

The case comes with a pair of 1400W PSU's in 1+1 redundancy... it will quite happily run off one of them without issues. I think the most I've ever seen it pull from my UPS during boot was about 1200W.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,176
Miner PSU from Banggood. No. Please don't.
Sorry but saw this and just had to reply.

My current system is a Dual Xeon Platinum 8253 (125W TDP), 256GB RAM, 2 LSI 3008 HBA's, Dual port 10GBe card, Sun F80 Flash Accelerator, 12 6TB SAS drives, 12 4TB SAS drives, 4 1TB SATA SSD's, 2 x 512GB M.2 NVME SSD's in a Supermicro CSE847 36Bay 4u case.

The case comes with a pair of 1400W PSU's in 1+1 redundancy... it will quite happily run off one of them without issues. I think the most I've ever seen it pull from my UPS during boot was about 1200W.
This is a pretty typical situation. For 12 disks, even a 1 kW PSU would be plenty in most situations I can imagine that don't involve crazy GPUs. If not smaller, depending on hardware specifics.
 

Deeda

Explorer
Joined
Feb 16, 2021
Messages
65
Sorry but saw this and just had to reply.

My current system is a Dual Xeon Platinum 8253 (125W TDP), 256GB RAM, 2 LSI 3008 HBA's, Dual port 10GBe card, Sun F80 Flash Accelerator, 12 6TB SAS drives, 12 4TB SAS drives, 4 1TB SATA SSD's, 2 x 512GB M.2 NVME SSD's in a Supermicro CSE847 36Bay 4u case.

The case comes with a pair of 1400W PSU's in 1+1 redundancy... it will quite happily run off one of them without issues. I think the most I've ever seen it pull from my UPS during boot was about 1200W.
Just be careful during brownouts, might be a different story then...
 

Deeda

Explorer
Joined
Feb 16, 2021
Messages
65
Power supplies aren't protection against brownouts, UPS's are. My whole setup is UPS backed.
That's fine for brownouts, what about extended outages? I hope you have some form of auto shutdown, eg, NUT!
 

Deeda

Explorer
Joined
Feb 16, 2021
Messages
65
So got the build finished... couple of photos attached. In the cable management excitement managed to lift up one of the SATA ports ffs :eek:

I gave TrueNAS Scale a red hot go, but was just too buggy and kept hitting issues. Running TrueNAS Core on top of Proxmox now.
 

Attachments

  • case1.png
    case1.png
    1.4 MB · Views: 111
  • case2.png
    case2.png
    1.8 MB · Views: 121
  • sataport.png
    sataport.png
    1.6 MB · Views: 109

andyjay777

Dabbler
Joined
Jan 31, 2022
Messages
27
So got the build finished... couple of photos attached. In the cable management excitement managed to lift up one of the SATA ports ffs :eek:

I gave TrueNAS Scale a red hot go, but was just too buggy and kept hitting issues. Running TrueNAS Core on top of Proxmox now.
I think you did a better job of cable management than me

Keen to hear how your drive temps are going. The bottom two drives closest to the power supply run 8-10 deg hotter for me (compared to the other 14). I added a fan where you have the 2.5 SSD, but that didn’t help significantly.

I also have a fan directed at my HBAs and network card… potentially (/probably) unnecessary.

Nice to see someone else running a lot of drives… makes me feel better
 

Deeda

Explorer
Joined
Feb 16, 2021
Messages
65
At the moment for the 12 SAS drives, the temps range from 35 to 41 degrees celsius. I have another 4 SATA drives right at the bottom of the case that you may have spotted in the second photo, but I gave up trying to connect them for the time being after getting a bit frustrated with the cable management, and having difficulties with the SATA power connectors being long enough to reach all the drives. I'll probably revisit that when I need the additional space.

I originally moved one of the front fans down lower, so that those bottom SATA drives would get airflow. I've left it where it is for the time being anyway.
 
Top