Critical error with boot volume after update

[havok2022]

Getting the following critical alert after latest update of 9.3.

CRITICAL: The boot volume state is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.

I'm pretty new to FreeNAS so I'm uncertain on how I should proceed. I went and activated the previous version and rebooted. No change. I deleted the latest partition and updated again with no change. Install verification passes. Any suggestions on how to proceed?

Code:

[root@freenas ~]# zpool status -v | more                                                                                           
  pool: bigpool                                                                                                                    
state: ONLINE                                                                                                                     
  scan: scrub repaired 0 in 3h57m with 0 errors on Sun Jan 18 05:57:59 2015                                                        
config:                                                                                                                            
                                                                                                                                   
        NAME                                            STATE     READ WRITE CKSUM                                                 
        bigpool                                         ONLINE       0     0     0                                                 
          raidz2-0                                      ONLINE       0     0     0                                                 
            gptid/e6ef5e3a-936d-11e4-9e88-0cc47a313e4e  ONLINE       0     0     0                                                 
            gptid/e74e1f56-936d-11e4-9e88-0cc47a313e4e  ONLINE       0     0     0                                                 
            gptid/e7aee413-936d-11e4-9e88-0cc47a313e4e  ONLINE       0     0     0                                                 
            gptid/e84387eb-936d-11e4-9e88-0cc47a313e4e  ONLINE       0     0     0                                                 
            gptid/e8b01d35-936d-11e4-9e88-0cc47a313e4e  ONLINE       0     0     0                                                 
            gptid/7e2662c7-938f-11e4-869d-0cc47a313e4e  ONLINE       0     0     0                                                 
                                                                                                                                   
errors: No known data errors                                                                                                       
                                                                                                                                   
  pool: freenas-boot                                                                                                               
state: ONLINE                                                                                                                     
status: One or more devices has experienced an error resulting in data                                                             
        corruption.  Applications may be affected.                                                                                 
action: Restore the file in question if possible.  Otherwise restore the                                                           
        entire pool from backup.                                                                                                   
   see: http://illumos.org/msg/ZFS-8000-8A                                                                                         
  scan: none requested                                                                                                             
config:                                                                                                                            
                                                                                                                                   
        NAME        STATE     READ WRITE CKSUM                                                                                     
        freenas-boot  ONLINE       0     0     1                                                                                   
          da0p2     ONLINE       0     0     4                                                                                     
                                                                                                                                   
errors: Permanent errors have been detected in the following files:                                                                
                                                                                                                                   
        //boot/kernel/ipmi.ko                                                                                                      
        //usr/local/lib/python2.7/site-packages/django/utils/termcolors.py  

 

[BigDave]

The drives of your pool appear to be fine but your boot device seems
to be failing, which drive in your machine is the one listed with checksum errors?
Which drive is da0p2 ??? Is this your boot drive?
Can you post the smart results for that drive?

 

[havok2022]

The drive listed is the USB drive for OS Boot. I can just backup the config file via System -> General -> Save Config, correct? Its a brand new Crucial Drive, and one of their better ones. I would be very surprised if its already failing. Not sure how to output the S.M.A.R.T. results, or if that drive even has that service running. I know my discs in the pool do.

 

[BigDave]

I can just backup the config file via System -> General -> Save Config, correct?

Correct.
From the shell, type this...

Code:

smartctl -a /dev/device name here

 

[havok2022]

Seems it can't detect the type of device.

Code:

[root@freenas ~]# smartctl -a /dev/da0p2                                                                                           
smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p8 amd64] (local build)                                                         
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org                                                        
                                                                                                                                   
/dev/da0p2: Unable to detect device type                                                                                           
Please specify device type with the -d option.                                                                                     
                                                                                                                                   
Use smartctl -h to get a usage summary                                                                                             
                                                                                                                                   
[root@freenas ~]# smartctl -a /dev/da0p2 -d                                                                                        
smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p8 amd64] (local build)                                                         
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org                                                        
                                                                                                                                   
=======> ARGUMENT REQUIRED FOR OPTION: d                                                                                           
=======> VALID ARGUMENTS ARE: ata, scsi, sat[,auto][,N][+TYPE], usbcypress[,X], usbjmicron[,p][,x][,N], usbsunplus, 3ware,N, hpt,L/M
/N, cciss,N, areca,N/E, atacam, auto, test <=======                                                                                
                                                                                                                                   
Use smartctl -h to get a usage summary

 

[BigDave]

try just da0 without the p2

 

[havok2022]

No dice. Seeing the same thing. :(

 

[Ericloewe]

USB flash drives don't generally support SMART in any way. Only those that are actually SATA SSDs behind a USB/SATA bridge do.

 

[BigDave]

Looks like you will have to replace your boot device and re-install Freenas.
Once your boot drive is operational, you can simply import your existing pool.
A couple of options regarding boot devices, first, to avoid what has occured
to you this time, consider using two usb drives in a mirror. Second option,
if the group of drives you have came from the same source, I would not trust
them.

 

[havok2022]

Thanks to all. I'm back up and running now. I considered the mirror, but I like that the board itself has USB built onto it so I dont have to worry about a USB stick hanging out the back for potential risk. I also rolled the dice on using the same USB. I've not had on of Corsair's higher end USB flash drives fail on me yet, and given the last update hung and was behaving oddly I thought it was likely a corrupted update. All appears to be well so far. If it happens again I'll replace the drive and consider a mirror.

 

[pschatz100]

You should replace the USB boot drive. USB sticks are just not very reliable. Even the good ones have issues, and where they tend to fail is when writing data - such as during an update. At the first sign of questionable activity, a USB stick should be replaced. Consider it a warning. Why would you want to risk your system?

 

[Jailer]

You should replace the USB boot drive. USB sticks are just not very reliable. Even the good ones have issues, and where they tend to fail is when writing data - such as during an update. At the first sign of questionable activity, a USB stick should be replaced. Consider it a warning. Why would you want to risk your system?

Yeah I have to agree with this post 100%.

Freenas did it's job and reported an error on your boot drive and you're just going to ignore it?

 

[danb35]

I like that the board itself has USB built onto it so I dont have to worry about a USB stick hanging out the back for potential risk.

My board also has a single USB A port on it, but all motherboards have USB headers on them. I bought a header adapter from Amazon to allow me to put the second USB stick inside my chassis. There are lots of styles of such adapters available; I'm sure you can find one that fits your system and use case.

 

[havok2022]

I picked up a few Cruzer Fits yesterday and the system has been powered down for a few days. I'll replace the USB when I have time and put a 2nd on the back. I picked up the small ones so they dont stick far out of the back. Are there any issues with these I need to be aware of?

 

[danb35]

One of my USBs is a Cruzer Fit and I haven't had any trouble with it so far. They seem to be well-regarded around here, from what I can see. But, again, it's simple enough for you to mount both USBs inside your case.

 

[pschatz100]

The small form factor is certainly the way to go. Personally, I would not buy a Cruzer Fit because they do not publish performance specs for it - which means they can use whatever parts from the "bin" they want. There are other choices. However, if you already have it then use it.

I do not buy components for which specs are not published. You can search the forum for my thoughts about flash drives.

Update** People seem to be having good luck with the Cruzer Fit.

 

[SandLake]

I have just encountered this error/message - can I use USB imaging software to create a new bootable USB key? Any recommendations?

 

[SweetAndLow]

Better to just reinstall 9.3 by following the directions in the manual.

 

[SandLake]

Thanks

 

[KitDavis]

I have a number of 9.3 servers that use USB drives for boot volumes. One of them developed this error a couple of months ago after an update. Checking the status of the boot devices indicated there were 17 checksum errors. It was at a remote location, so I crossed my fingers that it would last until I could schedule the replacement. A few days later I had too many console windows open and applied a new set of updates to the this box by mistake. I was sure that I was going to be taking an emergency run for an unscheduled repair as a result, but surprisingly not only did the system not fail during the update but once rebooted, the error was no longer present and the boot drive was clear of errors. The system has worked through a number of updates over the last 3 months with no problems.

A second system developed the same error last week (the boot device status showed 3 checksum errors) I waited for a couple of days until there were new updates available, applied them and again, the checksum errors disappeared. I plan on replacing all of the USB drives with small SSDs over the next couple of months, but I am not sure that this error is quite as critical as it seems.