freenas reports: One or more devices has experienced an error resulting in data corruption.

[ghostwolf59]

Guys and (girls :) ) - how do I identify the device(s) or files resulting in that freenas flag this... ?

WARNING: The volume STORE02X (ZFS) status is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.Restore the file in question if possible. Otherwise restore the entire pool from backup.
(image of message attached)

This drive holds my jails and they are all up and running yet freenas seem to identify some issues with this drive.

I have tried scrub but no change - do I need to restore the entire volume or...?

Perhaps worth mention that torrents I download to a torrent directory on this volume cant be copied across to another volume - when I do I get a "insufficient privileges/access error" reported by the windows client


cheers

 

[dlavigne]

Post the output of zpool status within code tags.

 

[ghostwolf59]

Post the output of zpool status within code tags.

Where can I see this - the volume status only show the volume(name)/stripe/adap2
and reported 30 errors at the top - no idea where I can see or report on those errors - is there a report I can run or...?



This is the output from zpool status -v

Code:

pool: STORE02X
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub repaired 0 in 0h37m with 30 errors on Fri Dec 12 21:56:06 2014
config:

NAME STATE READ WRITE CKSUM
STORE02X ONLINE 0 0 1
gptid/6301a77a-55ac-11e4-8129-001e8ce02ee0 ONLINE 0 0 4

errors: Permanent errors have been detected in the following files:

STORE02X/BitTorrent:<0x1b>
/mnt/STORE02X/BitTorrent/Guardians.of.the.Galaxy.2014.720p.BluRay.x264-SPARKS[rarbg]/Guardians.of.the.Galaxy.2014.720p.BluRa
y.x264-SPARKS.mkv

and once I deleted this torrent file and rerun the zpool command I get this... pointing to the torrent download folder/dataset I have defined - perhaps I need to drop this dataset and re-create it ?

Code:

pool: STORE02X
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub repaired 0 in 0h37m with 30 errors on Fri Dec 12 21:56:06 2014
config:

NAME STATE READ WRITE CKSUM
STORE02X ONLINE 0 0 1
gptid/6301a77a-55ac-11e4-8129-001e8ce02ee0 ONLINE 0 0 4

errors: Permanent errors have been detected in the following files:

STORE02X/BitTorrent:<0x1b>
STORE02X/BitTorrent:<0x32>

Update: Decided to kill the BitTorrent zfs dataset and once done freenas fell apart - spat the dummy and pumped out lots of errors onto the console before rebooting and then during reboot it stopped at the dbg> prompt
The only way to get nas up and running again was to remove the disk - then once up I first tried to remount the drive but as soon as I did that nas rebooted again and finally fell back into the dbg> prompt
Then decided to resilver the drive - that worked, but now I face the possibility to reinstall and configure all my plugins - currently in the process to restore the resilvered drive with a backup image, but still not sure if this would work in terms of starting jail etc - Will post an update once the restore finished and I tested it out

cheers

 

[ghostwolf59]

so the restore didn't work (almost expected this to happen)

But now I'm faced with another problem - I want to re-create my jails but when I try to add jail I get this error...

Code:

Request Method: GET
Request URL: http://freenas/admin/jails/jails/add/
Software Version: FreeNAS-9.2.1.8-RELEASE-x64 (e625626)
Exception Type: warden_exception
Exception Value:
The command /usr/local/bin/warden list -v failed: "Error: No jails found!
"
Exception Location: /usr/local/www/freenasUI/../freenasUI/common/warden.py in run, line 447
Server time: Sat, 13 Dec 2014 16:35:56 +0800
Traceback

Environment:

Software Version: FreeNAS-9.2.1.8-RELEASE-x64 (e625626)
Request Method: GET
Request URL: http://freenas/admin/jails/jails/add/


Traceback:
File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  107.  response = middleware_method(request, callback, callback_args, callback_kwargs)
File "/usr/local/www/freenasUI/../freenasUI/freeadmin/middleware.py" in process_view
  158.  return login_required(view_func)(request, *view_args, **view_kwargs)
File "/usr/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
  22.  return view_func(request, *args, **kwargs)
File "/usr/local/www/freenasUI/../freenasUI/freeadmin/options.py" in wrapper
  209.  return self._admin.admin_view(view)(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapped_view
  99.  response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/views/decorators/cache.py" in _wrapped_view_func
  52.  response = view_func(request, *args, **kwargs)
File "/usr/local/www/freenasUI/../freenasUI/freeadmin/site.py" in inner
  144.  return view(request, *args, **kwargs)
File "/usr/local/www/freenasUI/../freenasUI/freeadmin/options.py" in add
  378.  mf = mf()
File "/usr/local/www/freenasUI/../freenasUI/jails/forms.py" in __init__
  150.  addrs = guess_addresses()
File "/usr/local/www/freenasUI/../freenasUI/jails/utils.py" in guess_addresses
  539.  ipv4_addr = guess_ipv4_address()
File "/usr/local/www/freenasUI/../freenasUI/jails/utils.py" in guess_ipv4_address
  433.  get_ipv4_exclude_dict()
File "/usr/local/www/freenasUI/../freenasUI/jails/utils.py" in get_ipv4_exclude_dict
  101.  jails = w.list()
File "/usr/local/www/freenasUI/../freenasUI/common/warden.py" in list
  1060.  return self.__call(warden_list(flags, **kwargs))
File "/usr/local/www/freenasUI/../freenasUI/common/warden.py" in __call
  1021.  tmp = obj.run()
File "/usr/local/www/freenasUI/../freenasUI/common/warden.py" in run
  447.  raise warden_exception(msg)

Exception Type: warden_exception at /admin/jails/jails/add/
Exception Value: The command /usr/local/bin/warden list  -v failed: "Error: No jails found!
"
Request information
GET
No GET data

POST
No POST data

FILES
No FILES data

COOKIES
Variable Value
csrftoken 'wg389lcnceCa6x8tpemK6ygaV5EXXItH'
sessionid 'mzbq01yvvsyqnaf20cbiudunkp1xc505'
fntreeSaveStateCookie 'root%2Croot%2F168%2Croot%2F182%2F200%2F202%2Croot%2F182%2F207%2Croot%2F182%2F207%2F209%2Croot%2F182%2F188%2Croot%2F182%2F188%2F190%2Croot%2F183%2F213%2Croot%2F183%2F213%2F215%2Croot%2F187%2F217%2F219%2Croot%2F187%2F212%2Croot%2F187%2F205%2F207%2Croot%2F187%2F199%2F201%2Croot%2F187%2F193%2F195%2Croot%2F187%2F217%2Croot%2F183%2Croot%2F183%2F189%2F191%2Croot%2F183%2F198%2F200%2Croot%2F181%2Croot%2F187%2Croot%2F187%2F193%2Croot%2F198%2Croot%2F198%2F204%2Croot%2F198%2F204%2F206%2Croot%2F198%2F226%2Croot%2F198%2F226%2F228%2Croot%2F190%2F196%2F198%2Croot%2F190%2F201%2F203%2Croot%2F190%2F218%2F220%2Croot%2F141%2F174%2F176%2Croot%2F190%2Croot%2F186%2Croot%2F185%2Croot%2F190%2F218%2Croot%2F190%2F201%2Croot%2F188%2F194%2Croot%2F189%2F217%2F219%2Croot%2F189%2F195%2F197%2Croot%2F189%2Croot%2F184%2Croot%2F188%2Croot%2F195%2Croot%2F195%2F229%2Croot%2F195%2F229%2F231%2Croot%2F200%2F239%2Croot%2F200%2Croot%2F192%2Croot%2F193%2Croot%2F193%2F232%2Croot%2F193%2F232%2F234%2Croot%2F200%2F239%2F241%2Croot%2F201%2Croot%2F201%2F240%2F242%2Croot%2F197%2Croot%2F197%2F231%2Croot%2F203%2Croot%2F203%2F237%2Croot%2F203%2F237%2F239%2Croot%2F203%2F231%2F233%2Croot%2F203%2F236%2F238%2Croot%2F197%2F225%2F227%2Croot%2F197%2F230%2Croot%2F197%2F230%2F232%2Croot%2F203%2F257%2F259%2Croot%2F204%2Croot%2F204%2F237%2Croot%2F204%2F258%2Croot%2F203%2F236%2Croot%2F203%2F241%2F243%2Croot%2F203%2F257%2Croot%2F201%2F234%2Croot%2F201%2F234%2F236%2Croot%2F199%2Croot%2F199%2F227%2Croot%2F199%2F227%2F229%2Croot%2F156%2Croot%2F228%2Croot%2F189%2F200%2F202%2Croot%2F194%2Croot%2F194%2F205%2F207%2Croot%2F204%2F225%2F227%2Croot%2F219%2F250%2F252%2Croot%2F287%2Croot%2F287%2F298%2F300%2Croot%2F287%2F303%2F305%2Croot%2F287%2F308%2F310%2Croot%2F287%2F318%2Croot%2F287%2F318%2F320%2Croot%2F305%2Croot%2F305%2F321%2Croot%2F305%2F321%2F323%2Croot%2F310%2Croot%2F310%2F326%2F328%2Croot%2F310%2F331%2F333%2Croot%2F310%2F341%2F343%2Croot%2F310%2F336%2F338%2Croot%2F310%2F316%2Croot%2F310%2F321%2F323%2Croot%2F310%2F341%2Croot%2F310%2F326%2Croot%2F309%2F325%2F327%2Croot%2F309%2F315%2F317%2Croot%2F309%2F330%2Croot%2F308%2Croot%2F308%2F339%2F341%2Croot%2F308%2F329%2Croot%2F309%2F330%2F332%2Croot%2F309%2F325%2Croot%2F352%2Croot%2F278%2Croot%2F283%2F314%2Croot%2F283%2F314%2F316%2Croot%2F1%2F7%2Croot%2F1%2F8%2Croot%2F1%2F9%2Croot%2F1%2F10%2Croot%2F1%2F11%2Croot%2F1%2F12%2Croot%2F1%2F13%2Croot%2F225%2F235%2Croot%2F297%2F328%2Croot%2F297%2F328%2F330%2Croot%2F1%2F14%2Croot%2F1%2F15%2Croot%2F295%2F326%2Croot%2F295%2F326%2F328%2Croot%2F296%2F327%2Croot%2F296%2F327%2F329%2Croot%2F300%2Croot%2F300%2F331%2F333%2Croot%2F300%2F316%2F318%2Croot%2F304%2Croot%2F302%2Croot%2F304%2F335%2Croot%2F304%2F335%2F337%2Croot%2F304%2F315%2Croot%2F304%2F315%2F317%2Croot%2F308%2F324%2Croot%2F308%2F324%2F326%2Croot%2F289%2F329%2F331%2Croot%2F304%2F310%2Croot%2F304%2F320%2Croot%2F304%2F334%2Croot%2F304%2F339%2Croot%2F299%2Croot%2F304%2F339%2F341%2Croot%2F298%2Croot%2F302%2F337%2Croot%2F294%2Croot%2F293%2Croot%2F72%2F79%2F172%2F173%2Croot%2F72%2F79%2Croot%2F72%2F79%2F172%2Croot%2F72%2F79%2F172%2F181%2Croot%2F57%2F69%2Croot%2F297%2Croot%2F296%2Croot%2F296%2F317%2Croot%2F296%2F317%2F319%2Croot%2F296%2F322%2F324%2Croot%2F296%2F307%2Croot%2F296%2F307%2F309%2Croot%2F295%2Croot%2F295%2F301%2Croot%2F295%2F301%2F303%2Croot%2F276'
META
Variable Value
wsgi.multiprocess False
HTTP_REFERER 'http://freenas/'
REDIRECT_STATUS '200'
SERVER_SOFTWARE 'nginx/1.4.4'
SCRIPT_NAME u''
REQUEST_METHOD 'GET'
PATH_INFO u'/admin/jails/jails/add/'
SERVER_PROTOCOL 'HTTP/1.1'
QUERY_STRING ''
CONTENT_LENGTH ''
HTTP_USER_AGENT 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0'
HTTP_CONNECTION 'keep-alive'
HTTP_COOKIE 'csrftoken=wg389lcnceCa6x8tpemK6ygaV5EXXItH; sessionid=mzbq01yvvsyqnaf20cbiudunkp1xc505; fntreeSaveStateCookie=root%2Croot%2F168%2Croot%2F182%2F200%2F202%2Croot%2F182%2F207%2Croot%2F182%2F207%2F209%2Croot%2F182%2F188%2Croot%2F182%2F188%2F190%2Croot%2F183%2F213%2Croot%2F183%2F213%2F215%2Croot%2F187%2F217%2F219%2Croot%2F187%2F212%2Croot%2F187%2F205%2F207%2Croot%2F187%2F199%2F201%2Croot%2F187%2F193%2F195%2Croot%2F187%2F217%2Croot%2F183%2Croot%2F183%2F189%2F191%2Croot%2F183%2F198%2F200%2Croot%2F181%2Croot%2F187%2Croot%2F187%2F193%2Croot%2F198%2Croot%2F198%2F204%2Croot%2F198%2F204%2F206%2Croot%2F198%2F226%2Croot%2F198%2F226%2F228%2Croot%2F190%2F196%2F198%2Croot%2F190%2F201%2F203%2Croot%2F190%2F218%2F220%2Croot%2F141%2F174%2F176%2Croot%2F190%2Croot%2F186%2Croot%2F185%2Croot%2F190%2F218%2Croot%2F190%2F201%2Croot%2F188%2F194%2Croot%2F189%2F217%2F219%2Croot%2F189%2F195%2F197%2Croot%2F189%2Croot%2F184%2Croot%2F188%2Croot%2F195%2Croot%2F195%2F229%2Croot%2F195%2F229%2F231%2Croot%2F200%2F239%2Croot%2F200%2Croot%2F192%2Croot%2F193%2Croot%2F193%2F232%2Croot%2F193%2F232%2F234%2Croot%2F200%2F239%2F241%2Croot%2F201%2Croot%2F201%2F240%2F242%2Croot%2F197%2Croot%2F197%2F231%2Croot%2F203%2Croot%2F203%2F237%2Croot%2F203%2F237%2F239%2Croot%2F203%2F231%2F233%2Croot%2F203%2F236%2F238%2Croot%2F197%2F225%2F227%2Croot%2F197%2F230%2Croot%2F197%2F230%2F232%2Croot%2F203%2F257%2F259%2Croot%2F204%2Croot%2F204%2F237%2Croot%2F204%2F258%2Croot%2F203%2F236%2Croot%2F203%2F241%2F243%2Croot%2F203%2F257%2Croot%2F201%2F234%2Croot%2F201%2F234%2F236%2Croot%2F199%2Croot%2F199%2F227%2Croot%2F199%2F227%2F229%2Croot%2F156%2Croot%2F228%2Croot%2F189%2F200%2F202%2Croot%2F194%2Croot%2F194%2F205%2F207%2Croot%2F204%2F225%2F227%2Croot%2F219%2F250%2F252%2Croot%2F287%2Croot%2F287%2F298%2F300%2Croot%2F287%2F303%2F305%2Croot%2F287%2F308%2F310%2Croot%2F287%2F318%2Croot%2F287%2F318%2F320%2Croot%2F305%2Croot%2F305%2F321%2Croot%2F305%2F321%2F323%2Croot%2F310%2Croot%2F310%2F326%2F328%2Croot%2F310%2F331%2F333%2Croot%2F310%2F341%2F343%2Croot%2F310%2F336%2F338%2Croot%2F310%2F316%2Croot%2F310%2F321%2F323%2Croot%2F310%2F341%2Croot%2F310%2F326%2Croot%2F309%2F325%2F327%2Croot%2F309%2F315%2F317%2Croot%2F309%2F330%2Croot%2F308%2Croot%2F308%2F339%2F341%2Croot%2F308%2F329%2Croot%2F309%2F330%2F332%2Croot%2F309%2F325%2Croot%2F352%2Croot%2F278%2Croot%2F283%2F314%2Croot%2F283%2F314%2F316%2Croot%2F1%2F7%2Croot%2F1%2F8%2Croot%2F1%2F9%2Croot%2F1%2F10%2Croot%2F1%2F11%2Croot%2F1%2F12%2Croot%2F1%2F13%2Croot%2F225%2F235%2Croot%2F297%2F328%2Croot%2F297%2F328%2F330%2Croot%2F1%2F14%2Croot%2F1%2F15%2Croot%2F295%2F326%2Croot%2F295%2F326%2F328%2Croot%2F296%2F327%2Croot%2F296%2F327%2F329%2Croot%2F300%2Croot%2F300%2F331%2F333%2Croot%2F300%2F316%2F318%2Croot%2F304%2Croot%2F302%2Croot%2F304%2F335%2Croot%2F304%2F335%2F337%2Croot%2F304%2F315%2Croot%2F304%2F315%2F317%2Croot%2F308%2F324%2Croot%2F308%2F324%2F326%2Croot%2F289%2F329%2F331%2Croot%2F304%2F310%2Croot%2F304%2F320%2Croot%2F304%2F334%2Croot%2F304%2F339%2Croot%2F299%2Croot%2F304%2F339%2F341%2Croot%2F298%2Croot%2F302%2F337%2Croot%2F294%2Croot%2F293%2Croot%2F72%2F79%2F172%2F173%2Croot%2F72%2F79%2Croot%2F72%2F79%2F172%2Croot%2F72%2F79%2F172%2F181%2Croot%2F57%2F69%2Croot%2F297%2Croot%2F296%2Croot%2F296%2F317%2Croot%2F296%2F317%2F319%2Croot%2F296%2F322%2F324%2Croot%2F296%2F307%2Croot%2F296%2F307%2F309%2Croot%2F295%2Croot%2F295%2F301%2Croot%2F295%2F301%2F303%2Croot%2F276'
SERVER_NAME 'localhost'
REMOTE_PORT '56930'
wsgi.url_scheme 'http'
SERVER_PORT '80'
SERVER_ADDR '192.168.2.15'
DOCUMENT_ROOT '/usr/local/etc/nginx/html'
HTTP_X_REQUESTED_WITH 'XMLHttpRequest'
DOCUMENT_URI '/admin/jails/jails/add/'
wsgi.input <flup.server.fcgi_base.InputStream object at 0x8116c8b50>
HTTP_HOST 'freenas'
wsgi.multithread True
HTTP_CONTENT_TYPE 'application/x-www-form-urlencoded'
REQUEST_URI '/admin/jails/jails/add/'
HTTP_ACCEPT 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
wsgi.version (1, 0)
GATEWAY_INTERFACE 'CGI/1.1'
wsgi.run_once False
wsgi.errors <flup.server.fcgi_base.TeeOutputStream object at 0x8116f9cd0>
REMOTE_ADDR '192.168.2.155'
HTTP_ACCEPT_LANGUAGE 'en-US,en;q=0.5'
CONTENT_TYPE 'application/x-www-form-urlencoded'
CSRF_COOKIE u'wg389lcnceCa6x8tpemK6ygaV5EXXItH'
HTTP_ACCEPT_ENCODING 'gzip, deflate'

this message is displayed when I try to add a jail (currently reported as non present) - several of my plugins is still listed under plugins where they sit in a STOPPED state - when I try to delete any of these I get the same error message - so now I am unable to setup jail from scratch again along with reinstall the plugins I want - How do I resolve this one ????

cheers

 

[ghostwolf59]

Ok, another attempt to restore and at least I managed to get things to work - jail is recognized and starts nicely and I can activate my plugins. - Would be really interested to know how to resolve the issue of nas thinking there's a jail even though nothing is there which in effect prevent me from starting from scratch and start again - on this occasion I deem myself lucky to get this back online again - for a second I thought my only way out of this mess would be to reinstall nas - but surely there must be some other alternative available once you got into this situation. - I think I provided all info possible on this matter but please let me know if I missed something

cheers