Alder Lake / Raptor Lake Build Advice | P Core only CPU vs P+E Core CPU | TrueNAS Scale Angelfish vs Bluefin | MW34-SP0 W680 Motherboard

[Lipsum Ipsum]

Also if you got the time, would you be able to share your PC/Server build? it would be very interesting to know what route you went in terms of CPU, CPU Cooler, HDD/SSD & PC Case choice :)

I posted my pending build in my sig. It's actually kind of funny how similar our builds are as well as use-case. The significant hardware was easy to decide on. I can't believe the amount of time I went around and around trying to decide on a case. Definitely paralysis by analysis.

I really was hoping I could find something that looked closer to a Synology or QNAP with desktop/shelf sexiness, but had more DIY flexibility and could fit a ATX mobo. The NAS 8-bay K7 case was the closest I could find but looks like it was made in the 1980s. It claims it fits an ATX board, but I think that's a typo as the stated dimensions are only 9.6" square.

I finally just decided to go with the Fractal Design Define 7, but then I had to decide on the just the Define 7 or the Define 7 XL with solid side panels. Back and forth, was it going to sit on the floor or the desk behind one of my monitors? Oh look, the Phanteks Enthoo Evolv X also has a little spot for a mini-ITX I could add my desktop in the same case and free up some more room. Well screw that, lets go big or go home with the Thermaltake Core W200! Damnit why can't they make the Define 7 Nano hold 14 drives too!??1!one? Finally settled on the Define 7 (no XL) balancing cost, size, and drive capacity.

I'm holding off on the main pool drives to see what Black Friday/Cyber Monday might bring if anything. I may also press my luck with some alleged "manufacturer refurbished" drives from ServerPartsDeals. I have an existing non-NAS media server using Drive Pool that the new one will replace that's not quite full, so I'm not in a super big hurry.

I'm also not 100% committed to ultimately going with TrueNAS at the moment. Saying that isn't ban-worthy here is it? :) I'm MUCH more comfortable with Windows Server, Hyper-V, and the .Net world as a developer, so jumping ship to TrueNAS is a major overall change for me. I'm no stranger to the Linux and Unix world either though so that helps.

 

[Ericloewe]

Does this mean the HW recommendation guide should be updated? It only specifies i3's, Pentiums, and Xeons right now.

It could do with an update, but to address the specific point:

1. See post #14 above​
2. Pricing is nothing short of insane​
3. Availability is limited (to put it mildly)​
4. DDR5 ECC UDIMMs are still suffering from points 2 and 3 as well​
5. Server motherboards haven't been released yet (ASRock has announced stuff, but it's still marked "PRELIMINARY")​
6. The E-core stuff needs to soak longer​

 

[Ericloewe]

If you have 6+ disks running 24x7 (spin-down is not a good idea for lifetime), then saving 10-20 watts on board+CPU is not necessarily a game changer.

It's relevant. My systems with 6 disks each idle at around the 40-50 watt range. Of course, there's an easy way to cut 10 W very quickly - get a board without IPMI.
Point is, chasing the absolute lowest power consumption is often a Pyrrhic endeavor.

 

[KingKaido]

I posted my pending build in my sig. It's actually kind of funny how similar our builds are as well as use-case. The significant hardware was easy to decide on. I can't believe the amount of time I went around and around trying to decide on a case. Definitely paralysis by analysis.

Haha our builds & use cases are indeed similar, i definitely feel the 'paralysis by analysis' sentiment, there has been ALOT of overthinking with this Homer Server Build.

and yes the PC Case decision has been a hard one! Majority of pc cases these days don't account for 8+ HDDs, or they'll say its a 'quiet' case that's just a airflow focused mesh case (where they want you use lots of fans at low RPM, but with mesh everywhere the sound from HDDs will leak through like crazy).

I looked at BeQuiet's offerings but they didn't seem convincing based on the reviews, and other brands don't offer sound dampening panels, mostly tempered glass or just metal side panels (the noise emitted is a big consideration for me), so thats why i ultimately decided on the Fractal Design 7, i did want the 7 XL originally but its pretty massive and seeing as i don't even have that many HDD to fill half of the case currently, it doesn't make sense.

That 8-bay NAS Synology/ Qnap case looks cool but i feel those style of enclosures, trade hotswap functionality for lower overall airflow and more noise since the HDDs are at the front of the case (also its hard to know if it will be good at reducing HDD vibrations).

 

[KingKaido]

Also @Lipsum Ipsum i see you have the i5-13600k in your build, have you checked if the SuperMicro Motherboard has released a new BIOS update to allow the 13th Gen CPUs to work? it might be best to get a 12th Gen CPU for now or wait until they release a new BIOS.

 

[Lipsum Ipsum]

Also @Lipsum Ipsum i see you have the i5-13600k in your build, have you checked if the SuperMicro Motherboard has released a new BIOS update to allow the 13th Gen CPUs to work? it might be best to get a 12th Gen CPU for now or wait until they release a new BIOS.

I got everything set up in the Define 7 case. That thing is a BEAST and I'm glad I didn't go with the XL version. I would have needed another hernia repaired. Push the power button and...nothing happens. Oops. Forgot to turn the UPS on. Turned it up. Push the power button . And... nothing again. Dang it. Forgot to flip the power supply switch too. Turned that on. Nothing will stop me now. Pushed the power button one more time and...still nothing.

Bunch of double checking, swapping cables, testing power supply and cable voltages, unplugging everything except the essentials I finally get the CPU fan to spin for a split second before it just turning off. The BMC heartbeat LED at least now flashing on-off on-off on-off steadily, almost mocking me. Nan-nah nan-nah boo-boo...

Turns out I should have checked back in on this thread earlier to see your comment, and then double checked with SuperMicro to confirm. When I ordered it, I relied on this FAQ entry. Apparently "can support" does NOT mean "currently supports". Also, "with BIOS R2.0 or above" does NOT mean that BIOS version R 2.0 or above even exists yet, or expected to exist before mid-November at the earliest.

The other kicker I learned after chatting with support this morning is that, apparently, I'll need to acquire a 12th gen CPU in order to update the BIOS to use a 13th gen CPU. Or I can RMA it back to SM for them to flash it for me. The IPMI web interface can update the BMC firmware, but BIOS updates aren't supported. Not exactly surprising, but that would have been really helpful if that was mentioned somewhere.

Sigh. At least I'll have something to look forward to over next weekend the following weekend Thanksgiving! Maybe.

 

[Etorix]

The IPMI web interface can update the BMC firmware, but BIOS updates aren't supported.

Not even with a (paid) key?

Supermicro X10/X11 motherboard FAQ

Please note: This guide has been moved to the Resources section of the forum and is available there. Future updates will be posted there. This thread now contains the archive of the last version of the guide before it was migrated and also functions as the discussion for that Resource. The...

www.truenas.com

 

[Ericloewe]

Yeah, I think I even heard that BIOS flashing was included without a special key in X13 boards.

 

[Lipsum Ipsum]

Not even with a (paid) key?

Supermicro X10/X11 motherboard FAQ

Please note: This guide has been moved to the Resources section of the forum and is available there. Future updates will be posted there. This thread now contains the archive of the last version of the guide before it was migrated and also functions as the discussion for that Resource. The...

www.truenas.com

The UI does have a spot for adding a license key, however it doesn't have a text box. It asks for a file to upload. I don't know if this is something new as I didn't see that as the process previously.

If I inspect the HTML, it does have the option to select the BIOS to update, but it's hidden with CSS. Tweaking the CSS can show it, but I didn't attempt to do anything as I wasn't sure if that was indeed my problem or I'd bork the board in the process. It ends up that the bios file hasn't been released so it's kind of a moot point at the moment.

The support agent I was messaging with today also said that it wasn't supported due to Intel ME firmware limitations. I generally take a lot of what first-level support says with a grain of salty skepticism. However in this case, it also doesn't show up as a option here. No X13* boards appear. A X12SAE shows, but not the -F variant. And the X11SAE-F shows, but specifically mentions "No OOB BIOS Flash". So I'm presuming that it doesn't actually support it.

This is a "workstation" board with the W680 chipset and not a "server" board.

 

[Ericloewe]

Right, I've seen that warning on some workstation boards. I won't even try to speculate why, but it's annoying as all hell.

 

[Lipsum Ipsum]

Sigh. At least I'll have something to look forward to over next weekend the following weekend Thanksgiving! Maybe.

I just can't win with my W680 board. Picked up a i5-12600K I'll eventually use for a family member's computer. Pop it in, nothing. Still does the same thing where it turns on momentarily.

SuperMicro agreed that it'd be highly unlikely that two retail boxed CPUS with two power supplies would have identical problems turning on and that the board was likely DOA. They said to try to get a replacement with my original retailer (Provantage) as it'd likely be faster than an RMA through SuperMicro.

It was actually quite amazing. I think the call to SuperMicro and then to Provantage were BY FAR the easiest two "tech support" calls I've ever had make. I've had to go through longer initial holds just to talk to someone, let alone fix the issue. Both companies gave me zero hassles and I already have another board getting shipped before sending back the first. Sucks my board was DOA, but it happens. recommend both companies with their responses.

 

[Ericloewe]

Sucks about the board, but I'm glad it's on the way to being solved.

 

[oncdoc]

I just can't win with my W680 board. Picked up a i5-12600K I'll eventually use for a family member's computer. Pop it in, nothing. Still does the same thing where it turns on momentarily.

SuperMicro agreed that it'd be highly unlikely that two retail boxed CPUS with two power supplies would have identical problems turning on and that the board was likely DOA. They said to try to get a replacement with my original retailer (Provantage) as it'd likely be faster than an RMA through SuperMicro.

It was actually quite amazing. I think the call to SuperMicro and then to Provantage were BY FAR the easiest two "tech support" calls I've ever had make. I've had to go through longer initial holds just to talk to someone, let alone fix the issue. Both companies gave me zero hassles and I already have another board getting shipped before sending back the first. Sucks my board was DOA, but it happens. recommend both companies with their responses.

Was the board that you received functional? Now? I'm thinking of ordering the same board versus trying to find the gigabyte one

 

[Lipsum Ipsum]

Was the board that you received functional? Now? I'm thinking of ordering the same board versus trying to find the gigabyte one

The original board I had was not functional as far as the main processor. The BMC subsystem worked and I could log in to the IMPI web interface, but that's it. Provantage (retailer I bought it through) replaced it through their SM distributor and the replacement didn't have any issues.

It was disappointing at the time with additional waiting, but I wouldn't use my one case as any indication of overall quality, reliability, etc for the particular model board or SM products in general. Even the best of any manufacturer can have a dud. Freak cosmic ray, "quality" handling by fedex/UPS/minimum wage warehouse employee, tiny solder blob that snuck by QA inspection...

I still don't have my server fully built and operational yet. I've done some initial breaking in of the components and played with a few different configurations as I'm new to the TrueNas ecosystem.

I don't regret my purchase as I wanted something with the current generation of desktop processors, ECC, and some power when I need it. However I would say I'm a more than a bit disappointed with the lack of a few features. It doesn't support flashing the bios via the BMC, something SM claimed is a limitation of the management engine and not IMPI. SM indicated that the board supported Raptor Lake with a BIOS update in a FAQ entry, but at the time I bought it, that BIOS hadn't even been released yet. It since has been though. Flashing that bios however requires a 12th gen CPU in order to gain 13th gen CPU functionality.

All that's kind of minor though as once you get it running, you're not usually flashing BIOS all that often. But the biggest disappointment is the absolute crap support for fan speed control. It's got "Standard", "Full", "Optimal" and "Heavy I/O" settings. Anything but optimal sounds like a jet engine taking off with the Noctua 140mm fans I have. Optimal results in the fans falling to where the board thinks the fans stall, then goes up to full speed to restart them. Rinse. Repeat. After a few times it gives up and runs them full blast.

I know there's scripts that can emulate PID control for the fans, but it's asinine that in 2022 a $450 "workstation" motherboard can't do real fan curves in the bios or BMC. If I was using this as a desktop workstation, what it's designed for, in a corporate office environment, it would be totally unacceptable.

 

[oncdoc]

Oh man, thank you so much for sharing that experience. I appreciate that a lot. I think it'll be helpful for many here.

 

[saf]

Hi - first post here I believe. Wondering if anyone has a working X13sae-f? I have been trying to get one working for the past couple of days and figured I'd share what I have run into so far.

• I purchased my board the last week of December and it shipped with the 2.0 bios
• I am able to run the Intel i9 13900K without issues. I was expecting to have to update based on earlier comments but not the case.
• Installed Ubuntu server 20.04 easy enough. I was also able to install Windows Server 2019. Both off USB using 1 TB M.2 SSD.
• First issue - LSI 9300-8i adapter. Not able to see any drives. I moved the card to my windows desktop, rebooted, control c, got into card and it said it is running in IT mode. So that shouldn't be an issue.
  • Updated firmware anyway since it was old, move back to X13sae-f board, Ubuntu sees it but no drives.
  • Open ticket with Supermicro and response to my ticket (their support is nice, quick, and so far pretty spot on for me) they said the PCIe x16 slot on this platform doesn't support any storage controlled card according to Intel. So, our LAB didn't test any storage controller card on this platform.
• I connect up all drives using the SATA ports, power up, go into BIOS, and only see 1 drive.
  • Open ticket with supermicro and they say the drives are not on the supported/tested list. I am using WD 16 TB drives (Western Digital WD160EDGZ)
  • Incase the drive is bad or I damaged it I move it over to my current X11SSH-CTF server, take storage offline, swap a disk, and sure enough it sees the drive and starts resilver. Drive is good.

The good news is that recent boards shipped with 2.0 bios. Support is there for the latest Intel i9 13900k. Installed OS's easy enough but I'm sort of blocked at this point in time. According to Supermicro they are saying no storage support on the PCE x16 slots. I was planning on migrating from my current X11SSH-CTF to this as I needed more CPU compute for a few things including Plex. I also don't understand how it can see only 1 drive on SATA port 4 (WD 16 TB drive) and not on the others while also seeing a WD 8 TB Red on SATA port 1. Seems like there is a limit of some sort although I don't know how the firmware or whatever magic is going on for drive support - but I've hit it.

Supermicro support is great and helpful. I'm going to assume some of this is my ignorance for not catching on that it doesn't support storage cards nor did I check the disk support list. I've just never run into that before (granted I'm just home lab so limited in exp).

tl;dr curious if anyone else has the board, cpu, working with multiple drives. I've had no success.

 

[Ericloewe]

I'm going to assume some of this is my ignorance for not catching on that it doesn't support storage cards nor did I check the disk support list.

Unless Intel is going out of their way, there's no way for the PCIe slot to not support a specific class of devices. And since the controller shows up in the OS, it's cleared any meaningful hurdles that might have been in its way.
The fact that the SATA ports are also not working correctly suggests that something is up with either power, disks or cables/backplanes.

So let's try to examine this systematically:

1. Do the problems exist in all OSes? In particular, Windows 11 and Ubuntu 22.04 should both work without hiccups.
2. What's the cable situation like with these drives?
3. Are any other drives affected?
4. What's the power supply situation?
5. Does the controller work in a different machine?

 

[saf]

Unless Intel is going out of their way, there's no way for the PCIe slot to not support a specific class of devices. And since the controller shows up in the OS, it's cleared any meaningful hurdles that might have been in its way.
The fact that the SATA ports are also not working correctly suggests that something is up with either power, disks or cables/backplanes.

So let's try to examine this systematically:

1. Do the problems exist in all OSes? In particular, Windows 11 and Ubuntu 22.04 should both work without hiccups.
2. What's the cable situation like with these drives?
3. Are any other drives affected?
4. What's the power supply situation?
5. Does the controller work in a different machine?

I don't know about the PCIe slots to be honest only what supermicro support said. I checked the ticket again and they did say "First of all, the PCIe x16 slot on this platform doesn't support any storage controlled card according to Intel. So, our LAB didn't test any storage controller card on this platform". It really does say 'doesn't support'.

Thank you for the debug/examine systematically block - let me see if I can fill it by what I have done. Anything else I am happy to try.

• Do the problems exist in all OSes? In particular, Windows 11 and Ubuntu 22.04 should both work without hiccups
  • I've installed both Ubuntu 20.04 LTS and Windows 2019 (recently but was not able to test much with it today).
  • Ubuntu was the first OS installed and I did see it referenced in the logs. I also saw it mentioned doing a lspci |grep LSI in the output. I didn't save that message but can verify.
  • When windows 2019 boots I was not able to see the message saying hit control - c to enter controller. I see this message when I installed the same card on my Desktop Windows 10 PC. I rebooted, saw the message, control-C, and saw the card. It showed up as SAS3009(C0) with fw version 16.00.10.00
  • I looked at the Windows 2019 server and see other devices with the <!> indicating they need driver updates. Looks like I need to configure the network card to try and resolve this.
• What's the cable situation like with these drives?
  • With the LSI card I purchased CableCreation Internal HD Mini SAS (SFF-8643 Host) - 4X SATA (Target) Cable. I checked the LSI card/docs and believe I have the right cables.
  • With the SATA ports I used the cables supplied by Supermicro - standard red SATA. Nothing fancy.
• Are any other drives affected?
  • WD 16 TB I mentioned. WD 10 TB drive. It does see a WD (red/NAS) 8 TB. That drive is not on the supported / tested drive list yet it shows up. I only had one to test. The 16 TB WD drive only shows up on SATA port 4 nothing else.
• What's the power supply situation?
  • I thought it was good but I'm starting to suspect power. I have a new, not opened until I installed it in this build EVGA SuperNOVA 750 GM, 80 PLUS Gold 750W, Fully Modular, ECO Mode with FDB Fan. The cables are super short but I was able to get things connected.
• Does the controller work in a different machine?
  • I installed the controller on my windows 10 desktop. Was able to reboot, control-c, and get into the card. Verified it was in IT mode. Updated firmware, etc. I believe it is good although I did not connect a drive. If I did this it would probably be smart because that would confirm the cable I bought. I didn't think of that.

I was looking for another power supply but I do not have one to try. I could order another, doesn't hurt to have a spare, but it would be a few days before it can get here. Not sure if I said it but ty for the reply.

 

[Ericloewe]

I was not able to see the message saying hit control - c to enter controller.

This is irrelevant, that's the BIOS extension ROM software giving you an opportunity to enter the LSI setup menu. After the OS has booted, it's irrelevant - in fact, I typically advise to disable CSM and use UEFI only as much as possible (in UEFI mode, with the correct extension ROM flashed to the card, the system setup menu is extended with the same options you'd have in the LSI configuration thingy for legacy BIOS).

Do the problems exist in all OSes? In particular, Windows 11 and Ubuntu 22.04 should both work without hiccups

• I've installed both Ubuntu 20.04 LTS and Windows 2019 (recently but was not able to test much with it today).
• Ubuntu was the first OS installed and I did see it referenced in the logs. I also saw it mentioned doing a lspci |grep LSI in the output. I didn't save that message but can verify.
• When windows 2019 boots I was not able to see the message saying hit control - c to enter controller. I see this message when I installed the same card on my Desktop Windows 10 PC. I rebooted, saw the message, control-C, and saw the card. It showed up as SAS3009(C0) with fw version 16.00.10.00
• I looked at the Windows 2019 server and see other devices with the <!> indicating they need driver updates. Looks like I need to configure the network card to try and resolve this.

What about the disks? Seeing the controller is important, of course, but talking to the disks is the critical point here.

What's the cable situation like with these drives?

• With the LSI card I purchased CableCreation Internal HD Mini SAS (SFF-8643 Host) - 4X SATA (Target) Cable. I checked the LSI card/docs and believe I have the right cables.
• With the SATA ports I used the cables supplied by Supermicro - standard red SATA. Nothing fancy.

Can you try different cables or rotating them around, just to eliminate the cables as a source of problems?

Are any other drives affected?

• WD 16 TB I mentioned. WD 10 TB drive. It does see a WD (red/NAS) 8 TB. That drive is not on the supported / tested drive list yet it shows up. I only had one to test. The 16 TB WD drive only shows up on SATA port 4 nothing else.

And this behavior follows the drives and not the ports?

What's the power supply situation?

• I thought it was good but I'm starting to suspect power. I have a new, not opened until I installed it in this build EVGA SuperNOVA 750 GM, 80 PLUS Gold 750W, Fully Modular, ECO Mode with FDB Fan. The cables are super short but I was able to get things connected.

What I've done over the years is standardize on Seasonic fully modular PSUs, since their cables have been consistent for some 10 years now - that way I can just grab one off the shelf when needed, for any desktop-style system, and not worry about cables.

Does the controller work in a different machine?

• I installed the controller on my windows 10 desktop. Was able to reboot, control-c, and get into the card. Verified it was in IT mode. Updated firmware, etc. I believe it is good although I did not connect a drive. If I did this it would probably be smart because that would confirm the cable I bought. I didn't think of that.

Exactly, connecting disk(s) would be very helpful in narrowing this down.

 

[saf]

This is irrelevant, that's the BIOS extension ROM software giving you an opportunity to enter the LSI setup menu. After the OS has booted, it's irrelevant - in fact, I typically advise to disable CSM and use UEFI only as much as possible (in UEFI mode, with the correct extension ROM flashed to the card, the system setup menu is extended with the same options you'd have in the LSI configuration thingy for legacy BIOS).

What about the disks? Seeing the controller is important, of course, but talking to the disks is the critical point here.

Can you try different cables or rotating them around, just to eliminate the cables as a source of problems?

And this behavior follows the drives and not the ports?

What I've done over the years is standardize on Seasonic fully modular PSUs, since their cables have been consistent for some 10 years now - that way I can just grab one off the shelf when needed, for any desktop-style system, and not worry about cables.

Exactly, connecting disk(s) would be very helpful in narrowing this down.

Thank you for the feedback. Busy day at work so I didn't get a chance to work on this. However, I did boot the server and look at the bios in a bit more detail. I missed this previously but I was able to see the LSI card in the BIOS and select it. I shut it down, connected up 2 drives, and only saw one drive.

I need to try and move the card to my desktop PC and see if it does the same.


LSI card visible - must have missed this before:

Controller properties:

2 drives connected, only 1 drive visible / selection from menu