(Freenas 11.3-U3/4) Bootloop: panic: on mounting zfs pool

chippy94

Cadet
Joined
May 11, 2018
Messages
8
This is my hardware information:
ASRock c2750d4i mobo running P2.90 uefi bios
Atom c2750 cpu
32gb of ecc ram (4*8gb)

Hard drive pool drives and info:
Pool1:

5 x HGST 4tb NAS drives (raidz 1)
3 x HGST 8tb NAS drives (raidz 1)

OS
sandisk cruzer 16gb x 2

Hello,
On Tuesday 9/1 the power cut at my apartment. My Freenas (Originally running Freenas 11.3-U3) booted up and appeared to be working fine afterwards but yesterday (9/2) I noticed that it had become non-responsive.
I restarted it and went to the ipmi console redirect where I saw that it was endlessly boot-looping with the following ending:
1600685357701.png

Steps tried to resolve:
0) I physically detached the drives and was able to boot into Freenas web ui.
-I then exported my pool. Re-attached the drives and tried to re-import my pool. This failed as above.

1) I have booted into single user mode and manually imported the pool (pool1) which succeeds but throws a cannot mount error.
-When I did a zpool status on the pool imported in the command line (after importing) it notified me that one of the drives in the 3 drive raid had check sum errors. In case it was the checksum error log for that drive that was causing the problem I tried clearing the error. This has not helped.

2) I have already tried re-installing FreeNas on my sandisc cruzer fit 16gb drive (Now running FreeNAS-11.3-U4.1)
-In the new install I tried importing the pool again and then immediately ended up in the bootloop from above.

3) I have tried scrubbing the unmounted pool but no errors were found.

Any advice on steps to try to take to solve this problem? I'd love to hear that I've not lost all my data.
If I have a bad drive in any of my pools I don't mind replacing it but losing years of data would be unpleasant. =P

Best Regards,
Chris


Note: Post moved to storage section from general one. Hope this will attract the attention of people who can help.
 
Last edited:

Alecmascot

Guru
Joined
Mar 18, 2014
Messages
1,175
Does it boot if you remove the pool drives ?
 

chippy94

Cadet
Joined
May 11, 2018
Messages
8
Does it boot if you remove the pool drives ?
Yes it does.
I've removed the drives, booted, exported the drives and then reattached and tried to re-import them. Things go wrong on re-import. Initially I could import (but not mount) via the command line during the boot process using single user settings.
 

chippy94

Cadet
Joined
May 11, 2018
Messages
8
Updated spoiler tag in original post because it cut off the latter part of my post.
 

chippy94

Cadet
Joined
May 11, 2018
Messages
8
Managed to mount the pool in read only mode and now I'm saving the data to a hard drive array via SSH it's slow going but it's working. Does anyone have tips or suggestions for things to do once I've fully backed things up and decide to wipe the old pool and start over? Should I test some of the hardware or something?
 
Top