The volume vol0 (ZFS) state is DEGRADED

[Hubertus Feindler]

One or more devices has experienced an error resulting in data corruption. Applications may be affected.

[root@freenas] ~# zpool status -vx
pool: vol0
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub repaired 0 in 13h24m with 0 errors on Sun Jan 3 13:24:41 2016
config:

NAME STATE READ WRITE CKSUM
vol0 DEGRADED 0 0 267
raidz2-0 DEGRADED 0 0 534
gptid/82695e5f-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/83dc4ff5-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/84b6d8f2-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/8594258a-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/8673fb54-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/87555e48-95c8-11e5-aa04-0015177b16ec ONLINE 0 0 0
gptid/88344bf8-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors

errors: Permanent errors have been detected in the following files:

vol0/zvol0:<0x1>


[root@freenas] ~# tw_cli
//freenas> show

Ctl Model (V)Ports Drives Units NotOpt RRate VRate BBU
------------------------------------------------------------------------
c0 9650SE-16ML 16 11 11 0 1 1 WeakBat



//freenas> /c0 show

Unit UnitType Status %RCmpl %V/I/M Stripe Size(GB) Cache AVrfy
------------------------------------------------------------------------------
u0 SINGLE OK - - - 3725.28 RiW OFF
u1 SINGLE OK - - - 3725.28 RiW OFF
u2 SINGLE OK - - - 3725.28 RiW OFF
u3 SINGLE OK - - - 3725.28 RiW OFF
u4 SINGLE OK - - - 3725.28 RiW OFF
u5 SINGLE OK - - - 3725.28 RiW OFF
u6 SINGLE OK - - - 3725.28 RiW OFF
u7 SINGLE OK - - - 3725.28 RiW OFF
u8 SINGLE OK - - - 3725.28 RiW OFF
u9 SINGLE OK - - - 3725.28 RiW OFF
u10 SINGLE OK - - - 3725.28 RiW OFF

VPort Status Unit Size Type Phy Encl-Slot Model
------------------------------------------------------------------------------
p4 OK u0 3.63 TB SATA 4 - WDC WD40EFRX-68WT0N0
p5 OK u1 3.63 TB SATA 5 - WDC WD40EFRX-68WT0N0
p6 OK u2 3.63 TB SATA 6 - WDC WD40EFRX-68WT0N0
p7 OK u3 3.63 TB SATA 7 - WDC WD40EFRX-68WT0N0
p8 OK u7 3.63 TB SATA 8 - WDC WD40EFRX-68WT0N0
p9 OK u4 3.63 TB SATA 9 - WDC WD40EFRX-68WT0N0
p10 OK u5 3.63 TB SATA 10 - WDC WD40EFRX-68WT0N0
p11 OK u6 3.63 TB SATA 11 - WDC WD40EFRX-68WT0N0
p12 OK u8 3.63 TB SATA 12 - WDC WD40EFRX-68WT0N0
p13 OK u9 3.63 TB SATA 13 - WDC WD40EFRX-68WT0N0
p14 OK u10 3.63 TB SATA 14 - HGST HUS724040ALE640

Name OnlineState BBUReady Status Volt Temp Hours LastCapTest
---------------------------------------------------------------------------
bbu On Yes WeakBat OK OK 0 xx-xxx-xxxx


What can I do?

Hardware is Memory with no ECC Ram. The device is used as an ISCSI-Device attached to Xenservers.

 

[Mirfster]

NAME STATE READ WRITE CKSUM
vol0 DEGRADED 0 0 267
raidz2-0 DEGRADED 0 0 534
gptid/82695e5f-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/83dc4ff5-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/84b6d8f2-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/8594258a-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/8673fb54-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors
gptid/87555e48-95c8-11e5-aa04-0015177b16ec ONLINE 0 0 0
gptid/88344bf8-95c8-11e5-aa04-0015177b16ec DEGRADED 0 0 0 too many errors

Hardware is Memory with no ECC Ram

I am relatively new to FreeNas, but thinking the fact that there is no ECC memory and suddenly 6 of 7 drives having tons of errors is not a good thing.

Sorry, I can't help much, but will be watching this thread.

 

[Mirfster]

BTW, can you post system specs? I am sure if any of the others decide to help they will first want to know.

Forum Rules: https://forums.freenas.org/index.php?threads/forum-rules.22553/

System Configuration Information: Items marked with an asterisk “*” are required for a failure postings. However, all items are recommended and could simply be part of your tag-line.

1. * FreeNAS version (ex. “FreeNAS 9.2.1.6-Release 64-bit”)
   
2. * Hardware:
   
3. * Motherboard (Model)
   
4. * CPU (Model)
   
5. * RAM Size (in GB and model)
   
6. Hard Drives (Model), Quantity, and RAIDZ configuration
   
7. Add-On cards
   
8. For network connectivity, include details on your network hardware/configuration.

Then again, you may not get much assistance since your system memory is "out of bounds"...

Edit: Quoted wrong item regarding using non-ECC. Have to find it again...

 

[jgreco]

c0 9650SE-16ML 16 11 11 0 1 1 WeakBat

Hardware is Memory with no ECC Ram. The device is used as an ISCSI-Device attached to Xenservers.

So you've got a horrible RAID controller with a failing battery. Get rid of the 9650SE and replace it with a pair of 9211-8i's.

Your pool is quite possibly toasted. Back up anything you can right now. That level of error is usually pool fatal.

As for the hardware, get a decent hardware platform, any of the server-oriented boards from Supermicro are usually a good choice. Use ECC RAM.

 

[jgreco]

Also, we warned you that card was a really bad choice.

https://forums.freenas.org/index.php?threads/4tb-disks-attached-to-3ware-9550se.38970/#post-244893

 

[Hubertus Feindler]

1. * FreeNAS version: FreeNAS-9.3-STABLE-201512121950
2. * Hardware:
3. * Motherboard: ASUS Rock 970
4. * CPU (Model): AMD FX 8150 - 3,6 GHz Eight Core (FD8150FRGUBOX) Prozessor
5. * RAM Size (in GB and model) 32GB Kingston ValueRam DIMM in 8GB DDR3-1600 CL11
6. Hard Drives (Model), Quantity, and RAIDZ configuration: RAIDZ2 with 7 WDC WD40EFRX-68WT0N0, passed as SINGLE drives throuh the 3Ware Adapter
7. Add-On cards: 3Ware 9650SE-16ML with battery, Intel PRO/1000 PT Quad Port Server Adapter
8. local Network, 4 ISCSI Interfaces connected via DELL Powerconnect 5424

Hope this helps...

 

[jgreco]

Well, the PT Quad Port and the hard drives are salvageable. The rest should probably be scrapped.

Assume you meant ASRock 970. A desktop grade board, might be okay for a desktop but not appropriate to FreeNAS.

As previously indicated, the 9650 is virtually unusable for the application.