Same PSU in the two servers? No, they differ.
Identical components of the two servers:
- ESXI 6.5, latest patches
- FreeNAS 11.0, now 11.1 with 32GB RAM for VM and using PCI passthrough
- Currently, LSI 9300-8i HBA (3008 controller)
- Mitxure of Seagate 10GB Enterprise, IronWolf, IronWolf Pro drives (mixed drives to reduce chance of failures due to a bad batch of identical drives)
- Intel X550 dual ethernet to 10Gbps switch
- All drives burned in/tested for 1 week, RAM tested with 24 hours of memtest, CPU and it's temperature tested with 24 hours of mprime torture test
Differing components:
- Server (main) without timeout errors: drive temperatures 25-33C, X10SRA MB with E5-2696, 192 GB ECC RAM, 8x10GB drives in 4x10GB mirror, backplane/cables, EVGA 850W ATX PSU
- Server (backup) with timeout errors: drive temperatures 31-36C, X10SDV MB with D-1541, 128 GB ECC RAM, 6x10GB drives in 3x10GB mirror, backplane/cables, Silverstone SX600-G 600W Gold SFX PSU, initially using MB-based SATA ports with PCI pass-through and ACHI driver (getting timeout errors on all 6 drives)
- VMs (and disk use) patterns
Things I've changed on server with timeouts without success: SATA cables, bypassing backplane, SATA HBA/drivers
When you say that PSU is a "big one", do you mean a frequent cause of problems, perhaps due to signal integrity secondary by high frequency noise or lower frequency voltage changes? While I've heard that can happen, as well as problems with power cables compromised by poor connections (high or varying impedance), in my personal experience with over-rated wattage, top tier PSUs, I've not yet encountered a problem that I've been able to attribute to the power rails. Unfortunately, my case for the backup server can only accommodate 6 drives, or I'd just swap the drives between the two systems to see if the timeout errors stayed with the backup server's MB/case/PSU/power cables or followed the drives. I could try a different PSU in the backup server case. Corsair sells a 600W SFX power supply that I could place in the backup server, I'd be tempted buy another EVGA 850W ATX PSU since that is one of the few differences between the servers. However, the ATX-sized PSU would likely increase drive temperatures due to less air flow from front to back in the fairly small Liam PC-Q25B mini-ITX case.