22.02-RC.1-2 on HP ProLiant DL380e Gen8

Kernel75

Cadet
Joined
Dec 22, 2021
Messages
4
Good day to all!

I'm struggling to install TrueNAS Scale on an HP ProLiant DL380e Gen8 (64GB RAM, SmartArray P410 configured as HBA, 4x 16TB HDD + 1x m.2 120GB SSD in USB adapter connected to the internal USB port).

"Burning" the installer is not a problem: I tried several methods (plain USB, plain DVD, multi ISO USB with Ventoy); the installation always complete successfully but (of course there is a "but") at the reboot the server complains about "Non-System disk or disk error. replace and strike any key when ready".

This is a list of my attempts to boot it somehow:
  1. reinstalling several times: boot failed
  2. reinstalling to a smaller (64GB instead of 120GB) SSD on USB m.2 adapter: boot failed
  3. reinstalling to a standard 1TB SSD drive: boot failed
  4. reinstalling to one of the 16TB HDD: boot failed
  5. reset BIOS to factory settings, wipe boot disk then reinstall: boot failed
I tried to force boot from local disk after starting with SystemRescue CD, but the screen remains black (no zfs support in SystemRescueCD grub?).
The same USB/SSD adapter boot successfully when connected to my laptop (but it has a UEFI "bios").

The only way to make it boot was reinstalling to a different USB drive (Kingston DataTraveller 16GB); it worked but I cannot trust the USB flash drive.
I also tried the same installation on a different HP DL380e gen8 server (they should be paired for gluster replication when in use).

Random thoughts:
  1. GPT vs MBR should not be a problem: the 16GB USB flash drive looks identical (not considering partition/volume size) to 64GB or 120GB USB SSD
  2. other DL380s we have in the office boot with no problem Ubuntu 20 from large drives (16TB); items 3 and 4 on previous list were expected to work but boot fails
  3. USB Drive size should not matter: if I put an ISO file on the 120GB SSD or if I use Ventoy, the USB SSD boots with no problem
Any suggestion (apart changing the server!) is welcome.

Roberto
 

Kernel75

Cadet
Joined
Dec 22, 2021
Messages
4
The P410 is configured in HBA mode and, anyway, the server won't boot even with the P410 removed.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
"HBA mode" is unacceptable. I explain why in the linked article.
 

sybreeder

Explorer
Joined
Aug 15, 2013
Messages
75
On DL380e G8 only legacy - MBR mode will work. But I wouldnt trust P410 in HBA mode. I've used LSI 9211-8i as hba in that HP. But fans weren't happy.
Try boot flash it by rufus in MBR legacy mode
boot laptop/pc in legacy mode then install it in that USB.
Remember that you can set rear bays using AHCI mode. Just switch sata cable for rear bay from front connector to motherboard. and switch internal controller to AHCI.
 

ClassicGOD

Contributor
Joined
Jul 28, 2011
Messages
145
Like @jgreco wrote - even in HBA mode those HP controllers are not acceptable for ZFS.

When it comes to booting - Gen8 HP servers are not capable of booting from NVME as they are BIOS only systems. When it comes to booting from HDD it's probably an issue with your P410 configuration. You probably just need to go into the P410 setup utility by pressing F8 at appropriate time in post sequence and select a correct boot drive.
 

Kernel75

Cadet
Joined
Dec 22, 2021
Messages
4
Thank you all for your suggestions and explanations.
I agree with you: P410 is evil when using ZFS. The problem is that ZFS, in this scenario, is far from being involved because the system does not show the classical GRUB boot menu.

My question is: if I physically remove the P410 from the server (as I did), no other drives are connected apart the USB drive with a plain m.2 SATA SSD (not NVMe) and the server is able to boot from the installation media (whatever it is), why is it not able to boot up after a successfull installation on /dev/sda (the USB SSD drive)?

As far as I can imagine, it should be a boot loader problem; maybe I'm missing something because I'm using "plain" GRUB on a removable device while I should use something different.

Now I'm reading through https://github.com/ndeineko/grub2-bios-uefi-usb looking for ideas.
 

ClassicGOD

Contributor
Joined
Jul 28, 2011
Messages
145
I have no idea unfortunately. If i understand you correctly if you write installer ISO to your USB to m.2 it boots but if you install OS to the same device it fails to boot? That is bizarre. Some USB drives can present themselves differently (like USB attached SCSI etc) Check if after you install the OS on it the drive is not visible as storage controller in Boot Controller Order in BIOS.

Also I'm not sure about DL380e G8 configuration but in DL380p G8 the SmartArray controller is built in to the board. The thing you can remove is a FBWC (Flash-Backed Write Cache for the array). The way to "remove" the SmartArray controller is to diable it in BIOS. But this should not have any impact on your ability to boot from other devices.

What I use and recommend to boot my DL380p is official optical drive cable (I use ones designed for DL360 as they are more available and much cheaper where I am. AFAIK only difference is custom length) that plugs to the motherboard with SATA to slimline SATA adapter and standard SATA SSD. That way you boot directly for the on board SATA controller for best compatibility, you just need to stick the ssd somewhere - I use double sided tape on PSU cage.
 

Kernel75

Cadet
Joined
Dec 22, 2021
Messages
4
Thank you, ClassicGOD!

Your post pointed me in the direction to make it boot.

I disconnected the rear backplane from the P410 and connected it directly to the server MB; after setting the BIOS to use the internal controller as SATA AHCI, at least I've been able to install and start TrueNAS SCALE on the DL380e Gen8.

My next (and probably last!) attempt will be with a smaller SSD on USB: I tried 120GB and 64GB and they failed; I should have a 32GB m.2 SSD laying around on my desk at home for the Raspberry Pi; I will test it and report back.


P.S.: On the DL380e Gen8 the Smart Array P410 is a dedicated card installed in the PCI riser.
 

ClassicGOD

Contributor
Joined
Jul 28, 2011
Messages
145
I was thinking about the rear backplane but not every DL380e has one (DL380p unfortunately has no option for one :( ) and I didn't want to recommend a solution that I could only speculate about working.
 
Top