Suddenly lost 3 pools

namnnumbr

Dabbler
Joined
Jan 19, 2021
Messages
14
Last night, all 3 pools that I had running were no longer detected. I decided to export the pools and attempted to reimport, only to receive an I/O error.

When I attempt to view any attached drives `ls /dev/disk/by-id`, I only see my boot disk. This happens to be the only drive directly attached to the motherboard.

Each pool has its own backplane, so since all 3 pools are down, it's not a backplane issue (unless they all failed simultaneously). Additionally, the drives are a mix of ssd and hdd, and the hdd are from different lots, so it's not a simultaneous lot failure.

Since very non-boot drive is routed through an HBA and sata expansion card, it makes me wonder if my hba suddenly died somehow. If I run `lspci -nn | grep -i hba` nothing seems to be detected.
I've tried "unplugging and replugging" -- what else should I do to debug?
 
Last edited:

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
Watch the console messages during boot. If the HBA doesn't emit any console messages, it's a good chance it's dead. If there's another PCI-E port you could move it to, that would rule out a failed motherboard slot.
 

namnnumbr

Dabbler
Joined
Jan 19, 2021
Messages
14
Thanks for the suggestion. I was not seeing any console messages initially. I tried removing the sas expander from the mix for one of the breakout cables and the HBA came back (and drives were recognized). So I'm guessing the issue is with the expansion card OR the cables to it. I'm going to get some replacement cables to try as a next step.
 
Top