Server upgrade recommendation - stability required

[Lasermi]

Hello all,

First and foremost, I apologize if this or a similar topic is being discussed. I was looking through the different threads and I was somehow lost. I couldn't find a good fit for my specific use case. My truenas system is currently made of:

• Silverstone DS380 case (Micro ITX form factor supporting up to 12 drives)
• Asus P10S-I motherboard (Micro ITX including 6 SATA and one M.2 connector, ECC RAM and iKVM)
• Intel i3-8100T CPU (low TDP so as to keep cooling requirements under control in such a small and crowded case)
• 32GB ECC DDR4 RAM
• A relatively large and mixed amount of drives, ranging anywhere from 500GB to 8TB, SSD, HDD; ... for different usage / pools...
• A Toshiba RC100 NVMe M.2 drive for the System dataset.
• Some cheap low profile SATA controller with QUAD ASM1062 chip
• Two external eSATA cases with SATA port multipliers (similar to Silverstone TS431U, i.e. JMB321 based).

I have marked the important features for me in ( ).
This system was on the base perfect because it fulfilled my requirements, but it never ran stable. I always had strange boot issues (BIOS would hang up on boot unless IKVM remote control is connected, now IKVM is disconnected when power is turned on, disk drives have been randomly appearing and disappearing without an understandable logic, both in TrueNAS as in the BIOS, ...). Interestingly, the drives are randomly detected or not across all controllers, I couldn't find a solid logic. Currently, 4 drives always work and they are split across both the Intel C232 integrated controller as on some ASM1062 chips

The external cases have been even more problematic, so I stopped using them altogether even though the flexibility they could provide was for me important (I can reuse the case flexibly via eSATA on TrueNAS and via USB on Windows computers). I know they are generally not recommended and calling for disaster, but I love the flexibility of moving the case from a computer to another, hook up drives so as to import data to TrueNAS, ... I just want it to work stably on some very short occasions when importing data, not as a main pool.

After several years of increasing instability, I'm giving up. I want a stable system, as cost effective as possible (and thus reusing existing hardware as much as possible), but fulfilling my requirements:
- Micro ITX form factor motherboard to fit in the current case
- IPMI support
- Total of 14 SATA ports
- Support for SATA port multipliers (sorry for insisting...).

Can anybody suggest a MoBo/CPU that could fit and where I could maybe reuse RAM?
Does anybody know a good HBA with support for SATA multipliers? I have seen https://docs.broadcom.com/doc/pub-005851 where "Addressing of multiple SATA targets through an expander" is mentioned... but I'm unsure what this means.

Maybe I should try an ASRock Rack E3C246D2I with current CPU and RAM, add the LSI 9400-8i and see the results?

I'm even considering a QNAP system (TS-873A-8G) where I could eventually fit my RAM and drives, it supports ZFS via QuTS HERO... and which hopefully would support expansion via some QNAP TL-D800C, which I could then move from PC to NAS... It could even hopefully run TrueNAS Core.

Many thanks in advance for any help here...

 

[pschatz100]

I don't have an Asus P10S board, so I can't speak from personal experience. However, the CPU specs for that board (as posted on the ASUS site) do not include the i3-8100 processor. I'm surprised that it even boots. If it were me, I would purchase a low-end Xeon processor that is supported by the board.

As for TDP, all modern processors are reasonably energy efficient and NAS's don't stress the CPU very much. At the end of the day, I doubt you would would have any problem cooling a low end Xeon. However, if you are overly concerned by this then purchase a low TDP Xeon that is supported by the board.

As for adding support for extra drives, you need to use a proper HBA. Don't waste your time with SATA multipliers. There are tons of posts that discuss this issue.

 

[Lasermi]

Hey, pschatz100, thanks for your kind reply. You're absoluytely right and thanks fro picking this up. I built two similar systems and I mixed them up in my head. The MoBo is an Asus P11C-I.

 

[Lasermi]

Hey, pschatz100, thanks for your kind reply. You're absoluytely right and thanks fro picking this up. I built two similar systems and I mixed them up in my head. The MoBo is an Asus P11C-I. Today I saw there were some outstanding BIOS updates and now Truenas won’t boot, I get BTX halted. I will be absent for a week. When I come back home, I think I'm going to strip the whole system appart. I'll rebuild everything, drive by drive, component by component and do a fresh install. I'll report results but I'm slowly loosing my faith in this system.
Thanks again for your help!

 

[jgreco]

SATA port multipliers (similar to Silverstone TS431U, i.e. JMB321 based).

Well there's your problem. We have a nice article for you.

Multiply your problems with SATA Port Multipliers and cheap SATA controllers

In the last year or two, we've had a resurgence of users asking about SATA Port Multipliers and cheap SATA controllers. Please, do NOT use port multipliers, and use cheap SATA controllers only after extensive research. SATA controllers and SATA...

www.truenas.com

- Support for SATA port multipliers (sorry for insisting...).

Nope. Won't work. A port multiplier forces you to share the available bandwidth on a SATA port, and this will be flaky during ZFS scrubs and resilvers, exactly at the worst possible times where you NEED reliable SATA during those stressy activities. Don't do it. You'll be sorry! (best Grinchy singsong voice)

Does anybody know a good HBA with support for SATA multipliers? I have seen https://docs.broadcom.com/doc/pub-005851 where "Addressing of multiple SATA targets through an expander" is mentioned... but I'm unsure what this means.

It means using an SAS expander. See the SAS Primer.

Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

With the introduction of SAS 12Gbps, seems like "it's time" to do a braindump on SAS. Work in progress, as usual. History By the late '90's, SCSI and PATA were the dominant technologies to attach disks. Both were parallel bus multiple drop...

www.truenas.com

There is no such thing as an HBA with support for SATA port multipliers. An HBA, at least if we're limiting ourselves to true HBA's and not just a bunch of AHCI controllers glommed onto a PCB, which some places do make and do mislabel as an HBA, generally speaks SAS and is therefore incompatible with port multipliers.

add the LSI 9400-8i and see the results?

The LSI 9300-8i is a less expensive option and probably less of an unknown quantity.

- Micro ITX form factor motherboard to fit in the current case
- IPMI support
- Total of 14 SATA ports
- Support for SATA port multipliers (sorry for insisting...).

Have you considered something like the Supermicro X10SDV-7TP4F? It's not micro ITX but it is flex ATX. Has 20 SATA ports, 16 of which come from a Broadcom 2116.