Unsure of issue: "One or more devices experienced an unrecoverable error"

[gpsguy]

At least you haven't lost your wife, girlfriend, etc, due to a catastrophic data loss.


Sent from my phone

 

[thedirtmerchant]

Yikes. I almost want to hear that story.

 

[thedirtmerchant]

OK, so I've been reading various guides/hints/tips/etc on what I should have built from the get-go and it is patently obvious that I almost need to rebuild from the ground up.

I have copied all my data that I really really would cry about if I lost (pictures, home videos of the kids) onto an external HDDs.

So, on to the fun...

I'm trying to get an understanding of what options I should be considering for the RAID type I should use.
Should I go with RAID1 again? Have 2 3TB HDDs, then when I need to add more space, add 2 more HDDs?
Is there a definitive guide for RAID type (haven't seen it in my searches). I have the funds to put together the box and buy a couple HDDs, but don't currently have the funds to buy more HDDs than that right now. I will primarily using it as storage of pictures and videos that I do not want to lose. I will also be using it to store movies that I want to stream to my PS3.

 

[cyberjock]

Yikes. I almost want to hear that story.

Hey, I helped a couple recover their family album from a corrupted zpool after they had lost their 2 children in a fire. When I talked to the guy on Skype he said something like 'should I just go lay in the street right now and end this?' His wife was hysterical and certainly in tears when I told them the chance of recovery was very low IMO. I did get their data back though.

 

[thedirtmerchant]

Well...that's a positive story...kinda...data wise at least.

 

[thedirtmerchant]

Rut Roh. Bought the new HDD to replace the one that failed last night, with a plan to replace today.
While walking up this morning I got an email that my volume is degraded.
I rebooted since it was showing a bunch of errors and now I get:

freenas smbd[4701]: matchname failed on 192.168.0.143 and a bunch of Timeout on slot 28 port 0
and AHCI reset: device not ready after 31000 ms (tfd = 00000080).
and some other AHCI messages...

Can I safely replace the ada0 that failed?
and more importantly...how do I do that w/o access to the GUI?
Hopefully there is a simple fix to get the GUI functioning for a bit so I can remove/replace.

separate question: where do you guys store pics to post onto this site?
have a pic of the messages..

 

[danb35]

You can post pics in your message itself--click the button that says "upload a file" below the message composition box.

 

[thedirtmerchant]

Duh...was looking in the activity buttons above.


Can I safely replace the ada0 that failed?
and more importantly...how do I do that w/o access to the GUI?
Or how do I reestablish GUI access?
Hopefully there is a simple fix to get the GUI functioning for a bit so I can remove/replace.

 

[thedirtmerchant]

If it's too blurry, let me know. Shaky hands, what can I say...

 

[cyberjock]

So what you need to do is disconnect the failed disk and then bootup the system. If you don't know which disk is the bad one you'll have to plug in one disk at a time (so you don't mount the pool) and figure out which disk is the bad disk from smartctl.

 

[thedirtmerchant]

So what you need to do is disconnect the failed disk and then bootup the system. If you don't know which disk is the bad one you'll have to plug in one disk at a time (so you don't mount the pool) and figure out which disk is the bad disk from smartctl.

Once the failed disk is out, the IP should come back and then I just go through the regular drive replace steps?

 

[thedirtmerchant]

OK. 2 issues now.
I replaced the failed drive (shut down, remove old, replace w/ new, zpool replace command)
Issue 1:
When I do a zpool status, it is still in a degraded state
The Resilvering appears to have stopped (21M 2.5 hours ago...hasn't changed in an hour).
Can I restart the resilvering? Or did I do something wrong?


Issue 2:
Still no GUI IP Address. I can't figure out what the problem is.

At the "Console setup" I get the regular options, but the GUI IP info doesn't even show.

 

[thedirtmerchant]

This: https://bugs.pcbsd.org/issues/4027 looks like the solve to my issue for the name/hostname mismatch
but I don't know where the globalconfig is in the directory structure since I can only access it via the command line...

Thoughts?

 

[thedirtmerchant]

This: http://serverfault.com/questions/646794/freenas-web-ui-not-responding looks even more like my problem.
except, when django appears to be fine, it looks like nginx is a problem for me.
when I do a "service nginx restart" I get:

Performing sanity check on nginx configuration:
Segmentation fault (core dumped)

Should I just be getting a new USB stick and installing a new version of FreeNAS?

 

[cyberjock]

Umm... you are *all* over the map bro! Holy crap!

No clue why the GUI IP isn't showing. If it's supposed to be DHCP it's not getting an address (there's a billion reasons for that, so I can't even guess what it might be).

The name/hostname mismatch means that the hostname connecting to the server and the hostname that matches that IP doesn't match. It's not a big deal and you can simply turn that off unless you feel you need that kind of security (hint: you don't if you don't even understand the error... and that link doesn't instill me with confidence that you understand it).

The WebGUI can be unresponsive for a bajillion reasons. Someone else's issue is likely to be totally different than yours unless there is something *extremely* specific to your exact situation and theirs that is identical.

Hint: His problem is not related to your problem. His sanity check passed and his system actually tried to start nginx while yours falls flat on its face while doing the sanity check. Not related at all except that "it's broken".

No clue why you think you need a new USB stick. You provided no evidence whatsoever to even give a hint that your boot device might be to blame at all. In fact, unless ZFS is claiming lots of corruption in your boot device (which I would expect you to have mentioned) then its probably a safe bet that your USB stick is just fine.

No offense, but it really sounds like you are just throwing everything, including the kitchen sink, at the window to see what will stick. Hint: You're going to spend much more time picking up glass than solving your problems if you want to go that way.

 

[thedirtmerchant]

Umm... you are *all* over the map bro! Holy crap!

No clue why the GUI IP isn't showing. If it's supposed to be DHCP it's not getting an address (there's a billion reasons for that, so I can't even guess what it might be).

The name/hostname mismatch means that the hostname connecting to the server and the hostname that matches that IP doesn't match. It's not a big deal and you can simply turn that off unless you feel you need that kind of security (hint: you don't if you don't even understand the error... and that link doesn't instill me with confidence that you understand it).

The WebGUI can be unresponsive for a bajillion reasons. Someone else's issue is likely to be totally different than yours unless there is something *extremely* specific to your exact situation and theirs that is identical.

Hint: His problem is not related to your problem. His sanity check passed and his system actually tried to start nginx while yours falls flat on its face while doing the sanity check. Not related at all except that "it's broken".

No clue why you think you need a new USB stick. You provided no evidence whatsoever to even give a hint that your boot device might be to blame at all. In fact, unless ZFS is claiming lots of corruption in your boot device (which I would expect you to have mentioned) then its probably a safe bet that your USB stick is just fine.

No offense, but it really sounds like you are just throwing everything, including the kitchen sink, at the window to see what will stick. Hint: You're going to spend much more time picking up glass than solving your problems if you want to go that way.

Yup. Well aware I'm all over the map because...I have no idea what the problem is and only have enough technical knowledge to know it isn't working and find clues.
I reset the network connections and attempt to ping my router and get 0 response.
So I tried to reset anything related to the web services (nginx, django) and nginx returned as above.
If nginx won't start or restart (and since I have no idea how to truly figure the problem out) and the web service is part of the OS (right? wrong?) I figure...why not just start fresh with the USB stick/OS.
After all...a new OS should have a functioning web service and all settings defaulted. I could either do that, or just reset everything to factory settings...but I'm not sure what/why I should/should not do one over the other.
My data is on the drives and backed up separately. So this is what I have come up with.

You are probably right, I'll be picking up glass...but, unfortunately with my low level of technical knowledge and lack of time or base understanding to expand on, I am unable to figure out what the glass was beforehand or should have been. I have made many bad decisions in retrospect on this build (my first try at this). I did some general research, bought the parts, put it together, tinkered with settings and got it up and running for a couple years, all for less than $600 including the disks.

Honestly, with the general hardware guidelines and slightly better awareness of general maintenance, I could easily something much much better together and have somewhat better understanding
of the what/how/why of it all. But I do also question whether I should do that...or just spend the $ on a preset version that does not require the mid-level techie know-how or admin skills.

 

[thedirtmerchant]

Though I take no offense, I don't want to go with just throwing everything at the window...but what should I do?

 

[cyberjock]

Honestly, I think you need to:

1. Go back to basics and learn this stuff. People can't help you if you don't even understand the terminology we use and stuff. This is gonna take time, it's not a one week project and you are up and running. Your thought process for some of the stuff you are linking and talking about completely escape me. I don't feel like you have an understanding of the basics and the relationships between the different things you are working with, so we don't have a common ground until you can understand things at the basic level.
2. Buy one of those pre-made setups and use that (and hope to your favorite diety) that nothing goes wrong.

Keep good backups in either case. ;)

Not trying to be a jerk or anything. I've got your best interests at heart. In my noobie guide I said that you do have to learn this stuff or you're likely going to learn what you didn't know when you lose your data suddenly and unrecoverably. My opinion is you need to slow down and work less on trying to make your setup actually work and more about learning and understanding all the technology underneath. There is only 1 document that you really need to use FreeNAS properly in 99% of cases. The doc.freenas.org stuff. Running commands at the command line like you were trying to do is one of the fastest ways to see things blow up in your face unless you are an expert and understand what all those custom iXsystems scripts do that make their magic.

 

[thedirtmerchant]

Honestly, I think you need to:

1. Go back to basics and learn this stuff. People can't help you if you don't even understand the terminology we use and stuff. This is gonna take time, it's not a one week project and you are up and running. Your thought process for some of the stuff you are linking and talking about completely escape me. I don't feel like you have an understanding of the basics and the relationships between the different things you are working with, so we don't have a common ground until you can understand things at the basic level.
2. Buy one of those pre-made setups and use that (and hope to your favorite diety) that nothing goes wrong.

Keep good backups in either case. ;)

Not trying to be a jerk or anything. I've got your best interests at heart. In my noobie guide I said that you do have to learn this stuff or you're likely going to learn what you didn't know when you lose your data suddenly and unrecoverably. My opinion is you need to slow down and work less on trying to make your setup actually work and more about learning and understanding all the technology underneath. There is only 1 document that you really need to use FreeNAS properly in 99% of cases. The doc.freenas.org stuff. Running commands at the command line like you were trying to do is one of the fastest ways to see things blow up in your face unless you are an expert and understand what all those custom iXsystems scripts do that make their magic.

No worries about being a jerk. I am well aware of my limitations and unfortunately don't have enough time or drive to truly expand my knowledge enough to be a qualified user/admin.
What I don't like about the the pre-made setups is that I end up having to throw even more money at it than I would with a FreeNAS box
and the "hope to your favorite deity that nothing goes wrong" thing. Which seems to be less probable (something going wrong) if I was using FreeNAS and qualified.

Regardless of that...I would really like to just get this GUI back and the status of the pool to not be degraded and then move one to either a build or a pre-made purchase in the future.

 

[cyberjock]

Regardless of that...I would really like to just get this GUI back and the status of the pool to not be degraded and then move one to either a build or a pre-made purchase in the future.

That should be easy. Do a fresh install of FreeNAS 9.3, then import your pool. You'll have to resetup everything, but your data will be there and you'll just need to restore redundancy and do the actual FreeNAS setup after that.