Boot volume degraded

[avalon60]

Yesterday I updated FreeNAS to 9.10 and all went well. Today after a reboot I get this message:

Boot Volume Condition: DEGRADED One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state.

The boot volume or device is a USB2.0 stick I have used in the NAS box for a year or so now.

I installed a 2nd USB 2.0 sick as a mirror, but I'm not sure if it is mirroring.
When I click on status, I see:
freenas-boot degraded
mirror-0 degraded
1305785088643518938 unavailable
da0p2 online

If I select freenas-boot and mirror-0 , at the bottom of the screen it say 'Attach'
If I select the long number and da0p2 , it says 'Replace'

Does the Replace mean to remove da0p2 and replace with something else, another usb drive?

I have read about where it can be resilvered how does that work, and is there a way in freenas to do it.

What should I do or what is the best way to do what should be done.

Apologies for all the questions as this has just happened out of the blue with no warning.

 

[Robert Trevellyan]

A couple of screenshots would be helpful.

You would use Attach to add another device to the mirror. I don't think this is what you want.

You would use Replace to replace the selected device. I think this is what you want.

In either case, installing the new device first will simplify matters.

 

[avalon60]

So would it be a case of just installing another usb stick. I am thinking about a spare Patriot Supersonic 32Gb to use.
I got this email from my freenas box regarding the problem:

Checking status of zfs pools:
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
Backup_Data 928G 683G 245G - 18% 73% 1.00x ONLINE /mnt
CCTV 928G 17.2G 911G - 7% 1% 1.00x ONLINE /mnt
WinShare 7.25T 1.81T 5.44T - 2% 24% 1.00x ONLINE /mnt
freenas-boot 7.19G 3.45G 3.74G - - 47% 1.00x DEGRADED -

pool: freenas-boot
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://illumos.org/msg/ZFS-8000-2Q
scan: scrub repaired 0 in 0h4m with 0 errors on Wed May 18 17:49:37 2016
config:

NAME STATE READ WRITE CKSUM
freenas-boot DEGRADED 0 0 0
mirror-0 DEGRADED 0 0 0
gptid/3f850095-85e5-11e4-8708-001f16a89eef ONLINE 0 0 0
13057850883643518938 UNAVAIL 0 0 0 was /dev/gptid/ba4ca184-875e-11e4-a252-0cc47a0c1eef

errors: No known data errors
This is a screenshot of the boot status.

 

[Robert Trevellyan]

So would it be a case of just installing another usb stick

Pretty much. The challenge is figuring out which stick went bad. You might have to do that by trial and error.

Kingston Data Traveler SE9 8GB USB Boot drive
Sandisk Cruzer Blade 16GB USB Mirrored Boot Drive

What's going on here?

 

[avalon60]

Pretty much. The challenge is figuring out which stick went bad. You might have to do that by trial and error.

What's going on here?

How do you mean? I used the 8gb Kingston in the NAS box on a USB port on the board. The Sandisk Cruzer was just plugged in to a socket on the outside of the case. This was my attempt at morroring the 2 drives,. so I don't know.

 

[Robert Trevellyan]

Sorry, I was confused, I read "Sandisk Cruzer Blade 16GB USB Mirrored Boot Drive" as "two Sandisk ..."

 

[avalon60]

Sorry, I was confused, I read "Sandisk Cruzer Blade 16GB USB Mirrored Boot Drive" as "two Sandisk ..."

So from my screenshot, does that give any indication of which driver is which, as it doesn't make it clear enough for me.

 

[Robert Trevellyan]

No, the screenshot doesn't help, but you might be able to figure it out by matching device names with dmesg output. In other words, the one that isn't da0 is the one you need to replace.

EDIT: come to think of it, this demonstrates an advantage of using two different models of USB stick. When they're identical, so is the output in dmesg and the only way to tell them apart is by serial number, which is not always reliable with USB sticks.

 

[avalon60]

Ok so after I find out which one it is, and I'm guessing it is the one inside on the board , what do I do if my system won't boot. I am having a problem trying to install FreeNAS 9.10 onto another USB drive. I have created a separate thread for this problem.

 

[Robert Trevellyan]

what do I do if my system won't boot

Clean install and restore your saved config.

The system boots now, albeit with a degraded boot pool, right? No reason to think it won't after you replace the faulty device.

 

[avalon60]

Yes it did , but what I ended up doing, after going round in circles, was to get FreNAS 9.3 on another USB 16Gb drive. I booted from that and then uploaded my previously saved backup.

I actually did try the same with FreeNAS 9.10 but when I went to upload the backup, I got an error message saying it could not upload said backup because what was i=on the USB drive was newer than the backup. Ok, but I did update to FreeNAS 9.10 the day before.

I am now getting an error with the boot status, saying: unable to load /api/v1.0/system/bootenv/status status: 500.
Whatever that means.

 

[Robert Trevellyan]

OK, well at least you have a working system.

 

[avalon60]

Yes all is well again now, and I have this time properly mirrored another USB drive, and shows up in the Boot Status

 

[ChiknNutz]

I just had this happen to me today (first time). Is there a way to fix the USB stick or is it a physical problem? What is it that actually goes wrong/bad with this stick? Thanks.

 

[tvsjr]

It's usually a physical problem. The sticks (and any flash memory, including high-dollar data center SSDs) have a certain endurance - each cell can only be rewritten a certain number of times. USB sticks aren't known for having phenomenal endurance (nor quality control)... over time, they tend to fail.

Personally, I very much prefer two small SSDs. I personally use two Intel 320 SSDs that I grabbed off eBay for $20/ea.

 

[Bidule0hm]

Even one SSD is far better than two mirrored USB sticks so when you'll be tired of replacing the USB stick(s) go on ebay and hunt for a small size SSD which will be cheap ;)

 

[ChiknNutz]

For now, I plan to just replace the offending USB stick. I've read up on how to identify which is the offender, but I am still not sure how to really tell. I am not much of a user of the OS other than when I first set this thing up over a year ago, so I've kinda forgotten much that I had originally learned. Anyway, I found a reference to using this command to find which drive is bad: dd if=/dev/da0 of=/dev/null
However, I really don't know exactly what I am doing with that. I ran that exact command in a SSH shell but rec'd nothing, just sat there. Any add'l help is appreciated.

Forgive my ignorance, but why is one SSD boot drive better than two USB sticks? I have also read up on this and see it pretty well divided opinion-wise on which is a "better" setup. To me, the mirrored redundancy of the two drives trumps the fact that the SSD will last longer. Can you have mirrored SSD boot drives?

 

[tvsjr]

Real SSDs are designed for the job... USB sticks aren't really intended to run an OS from. So, a real SSD has substantially better longevity.

You can absolutely do mirrored boot devices, assuming you have enough ports and power to run the drives. I do exactly this.

 

[Bidule0hm]

Yep, a SSD is far more resilient, especially on writes, than an USB stick ;)

Divided opinion? I can't remember if I saw that, but I can see that after replacing my USB stick twice in less than two years I didn't had any problem with the SSD since I put it here and I'm not the only one. It should outlast the server by at least an order of magnitude. The SSD will be more expensive at first (but even a 32 GB SSD is plenty so you can find one at relatively low price) but less expensive in the long term.