FreeNAS stuck in boot/crash loop with disks connected to HBA – Supermicro X11SCH-F + LSI 9207-8i

[intelligence]

Some backstory:
My first FreeNAS build died on me a couple of weeks back. I suspect a failing motherboard. The system would reboot occasionally and finally it did not boot at all. Tried another PSU and switched around the RAM and tried just one stick at a time (out of four). It wouldn't boot and I could not get into BIOS even.

I decided to get new parts as I figured I was due for an upgrade anyway, so here we are today.

Earlier today I booted the system up barebones, without any disks in place. I booted my previous USB with FreeNAS on it, exported my config and proceeded to install a fresh copy of FreeNAS on a M.2 drive and importing my config. All seemed to work fine at this point.

I've now finished mounting all drives and connecting them to the motherboard and I'm facing an issue. FreeNAS won't come online, it's constantly rebooting when it tries to mount/open the disks in the booting phase. I'm curious if this can be fixed or if I'm out of luck and need to rebuild my media collection.

As this is the first time a FreeNAS system fails on me and I have to migrate to new hardware it might be that I've done some sort of rookie mistake (moving from an Asrock board with built in SAS controller to a new board without built in SAS, I've got an external card now, flashed to IT mode – could that cause issues for example?).

 

[intelligence]

If I disconnect the cable from the raid controller I get FreeNAS up and running.

 

[Samuel Tai]

You could have a backplane issue, or a SAS cable issue from the HBA. How are the drives connected to the HBA?

 

[intelligence]

It's a SFF-8087 to SATA included with the HBA. The seller state that the card is flashed to IT mode, I haven't actually verified that.

HBA is a LSI 9207-8i

Picture from seller is attached.

 

[Samuel Tai]

Did you burn in the card with blank disks, instead of trying to move your current pool wholesale? You might've gotten a fake card.

 

[intelligence]

I have not. Don't have any blank disks at my disposal.
Tried another cable, same problem.

Could there be a potential issue that all disks were connected via built in HBA before? Now I connect 6 disks via sata on the board and remaining 4 via external HBA.

Doesn't the mps part here indicate that the HBA is present and in IT mode?

 

[intelligence]

Wondering if maybe this could be related: https://www.ixsystems.com/community/threads/freenas-11-3-sas3008-probable-driver-issue.82174/ although not same exact issue.

 

[intelligence]

Trying to boot with 11.2 U8 renders these errors (no reboot though).

 

[intelligence]

So now I tried with everything connected, including the SAS controller, but I unplugged all drives from the controller and started testing them one by one:

First drive connected: FreeNAS boots
First and second connected: Fails
First and third connected: Fails
First and fourth connected: Fails

Fourth drive connected: FreeNAS boots
Fourth and third connected: Fails

Third drive connected: FreeNAS boots

Second drive connected: FreeNAS boots

I'm thinking there's an issue with the SAS controller... HW issue? FreeNAS issue? ZFS issue? Anything else that could cause it?

 

[Samuel Tai]

In all likelihood, you may have received an RMA return fobbed off as a working card.

 

[intelligence]

sas2flash -list
LSI Corporation SAS2 Flash Utility
Version 16.00.00.00 (2013.03.01)
Copyright (c) 2008-2013 LSI Corporation. All rights reserved

No LSI SAS adapters found! Limited Command Set Available!
ERROR: Command Not allowed without an adapter!
ERROR: Couldn't Create Command -list
Exiting Program.

Hmm!

 

[intelligence]

Tried the SAS controller on a Windows machine just now, there it shows up like it should..

Where does that leave me? Compatibility issue with my motherboard?

 

[intelligence]

FYI mobo on FreeNAS system is a Supermicro X11SCH-F.

 

[Samuel Tai]

The other possibility is the PCI-E slot you used on the X11SCH-F is bad.

 

[intelligence]

The other possibility is the PCI-E slot you used on the X11SCH-F is bad.

I'll test this as soon as possible, went on vacation today. In the meantime, maybe move this thread to "Hardware"? Perhaps seems more appropriate and might get more eyes/thoughts on my issue.

 

[Samuel Tai]

OK, I moved the thread per your request.

 

[intelligence]

Tested the other PCI-E slot now:

sas2flash -list LSI Corporation SAS2 Flash Utility Version 16.00.00.00 (2013.03.01) Copyright (c) 2008-2013 LSI Corporation. All rights reserved No LSI SAS adapters found! Limited Command Set Available! ERROR: Command Not allowed without an adapter! ERROR: Couldn't Create Command -list Exiting Program.

Makes me wonder if LSI 9207-8i is not supported on this particular mobo..

 

[intelligence]

Reflashed the card via this guide.

Noticed some differences, this is with flash from seller (done via sas2flash.exe):

This is after flash with sas2flash.efi:

Notice that Bios Version and UEFI BSD Version is populated now.

Upon booting the system, this is present, which it wasn't before:

However, the system still get stuck in a loop when disks are connected to the HBA. Unplug all disks and it boots into FreeNAS, but sas2flash -list still shows no adapters found when SSH'ing into the box.

 

[intelligence]

Lol.. If I run sudo sas2flash -list the HBA is shown in FreeNAS aswell.. Guess fingers crossed that the HBA is borked and nothing else.

 

[Pilaarbijter]

Hey intelligence,

Did you resolve the issue?