Broadcast storm causes FreeNAS to reboot, ZFS volume now UNKNOWN

Status
Not open for further replies.

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
Uh.. you can't rebuild disks or do anything to a zpool that won't mount.. and yours isn't mounting.

Step one.. get the zpool to mount.

Step two.. do other stuff as appropriate.
 

scurrier

Patron
Joined
Jan 2, 2014
Messages
297
OK, I removed the disk and FreeNAS automatically remounted the zpool firstvol.

Then, I followed the FreeNAS manual directions to replace the disk via the GUI. It is resilvering now...
 

BigDave

FreeNAS Enthusiast
Joined
Oct 6, 2013
Messages
2,479
OK, I removed the disk and FreeNAS automatically remounted the zpool firstvol.

Then, I followed the FreeNAS manual directions to replace the disk via the GUI. It is resilvering now...
Sweet! WTG cyberjock:D
 

scurrier

Patron
Joined
Jan 2, 2014
Messages
297
Yes, so far, WTG @cyberjock and also @fta who earlier made the same suggestion!

I will save the champagne until it completes successfully. 11% right now.
 

scurrier

Patron
Joined
Jan 2, 2014
Messages
297
Champagne all around!
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
So did it complete resilvering successfully?
 

scurrier

Patron
Joined
Jan 2, 2014
Messages
297
Yes it's done resilvering. Thanks for your help!

I'm scrubbing now.

I ran the problem disk through Seagate's SeaTools SMART utility to get a code for RMA and obviously the disk failed and I got the code. I will try to wipe the disk before sending it. Is there any issue with plugging it back in? Is there a risk it will conflict since it used to be in my pool?

Any further suggestions?

And does anyone know how my broadcast storm could have possibly had anything to do with this? I'm at a loss.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
As long as you can do a "zpool status" and the bad drive is no longer listed before you plug the drive in, things are okay. That verifies that the disk is permanently removed from the zpool.

I doubt the broadcast storm had anything to do with it. What is likely at fault is the fact that the system spontaneously rebooted.

To be honest, if the situation had been different, I'd have wanted to get all of the disks sent to iXsystems for analysis of what actually went wrong during the reboot. Clearly *something* was totally messed up. ZFS should have gracefully recovered from the failed disk, but clearly didn't. I'm not sure exactly why. It could have been some weird behavior that was the reboot with the broadcast storm. I just don't know. If it had been a new pool with only test data I would have asked if I could get the disks sent to iXsystems. But I realize that home users are a little less than reluctant to mail off their personal data, let alone their zpool, to some company for analysis.

I'd say that if the scrub comes back clean then all is good again. ;)

I try to make it a high priority to remove any failed disk from the server whenever I can. It sounds like things went from okay to bad pretty quickly, so there may not have been much you could have done anyway.
 

scurrier

Patron
Joined
Jan 2, 2014
Messages
297
Scrub complete with zero errors. :D
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,525
Yay. Count your blessings and keep those backups more up to date. ;)
 
Status
Not open for further replies.
Top