CRITICAL: The volume array (ZFS) status is DEGRADED

[Adi]

Hi folks,

I've inherited a FreeNAS box that I had recently managed to upgrade to FreeBSD 9.2-RELEASE-p15 (FREENAS.amd64)

I can't upgrade it to newer versions due to this bug : https://forums.freenas.org/index.ph...semi-fix-for-mountroot-issues-with-9-3.26071/

Anyway, I just recently logged in to the box and it is displaying the message that my array is degraded.

I think that 2 drives were replaced in the past but they were never properly replaced so that FreeNAS still thinks they're gone.

Also, is it normal that i get a "Segmentation fault (core dumped)" error when running "gpart show" ?

How do I fix this situation ?

[root@nas] ~# zpool status -v
pool: array
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://illumos.org/msg/ZFS-8000-2Q
scan: scrub repaired 0 in 0h36m with 0 errors on Sun Dec 6 00:36:21 2015
config:

NAME STATE READ WRITE CKSUM
array DEGRADED 0 0 0
mirror-0 ONLINE 0 0 0
gptid/213f8c5d-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
gptid/21a56c69-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
gptid/58129098-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
gptid/587499a2-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
gptid/67182535-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
gptid/678276de-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
mirror-3 ONLINE 0 0 0
gptid/7304967d-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
gptid/735f7d0a-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
mirror-4 DEGRADED 0 0 0
gptid/7ef2b78c-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
5495501265892037153 REMOVED 0 0 0 was /dev/gptid/11b99a97-c105-11e3-8302-5404a68a8f64
mirror-5 ONLINE 0 0 0
gptid/8aee2eab-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
gptid/8b5200c0-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
mirror-6 ONLINE 0 0 0
gptid/96d6c64c-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
gptid/973bb88c-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
mirror-7 ONLINE 0 0 0
gptid/a2bf0534-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
gptid/a3243d10-069a-11e3-bc30-5404a68a8f64 ONLINE 0 0 0
cache
1992289467543289864 UNAVAIL 0 0 0 was /dev/gptid/701a6ef3-3f88-11e3-abdc-5404a68a8f64

errors: No known data errors
[root@nas] ~# camcontrol devlist
<HPT DISK 0_0 4.00> at scbus7 target 0 lun 0 (da0,pass0)
<HPT DISK 0_1 4.00> at scbus7 target 1 lun 0 (da1,pass1)
<HPT DISK 0_2 4.00> at scbus7 target 2 lun 0 (da2,pass2)
<HPT DISK 0_3 4.00> at scbus7 target 3 lun 0 (da3,pass3)
<HPT DISK 0_4 4.00> at scbus7 target 4 lun 0 (da4,pass4)
<HPT DISK 0_5 4.00> at scbus7 target 5 lun 0 (da5,pass5)
<HPT DISK 0_6 4.00> at scbus7 target 6 lun 0 (da6,pass6)
<HPT DISK 0_7 4.00> at scbus7 target 7 lun 0 (da7,pass7)
<HPT DISK 0_8 4.00> at scbus7 target 8 lun 0 (da8,pass8)
<HPT DISK 0_10 4.00> at scbus7 target 10 lun 0 (da10,pass10)
<HPT DISK 0_11 4.00> at scbus7 target 11 lun 0 (da11,pass11)
<HPT DISK 0_12 4.00> at scbus7 target 12 lun 0 (da12,pass12)
<HPT DISK 0_13 4.00> at scbus7 target 13 lun 0 (da13,pass13)
<HPT DISK 0_14 4.00> at scbus7 target 14 lun 0 (da14,pass14)
<HPT DISK 0_15 4.00> at scbus7 target 15 lun 0 (da15,pass15)
< Patriot Memory PMAP> at scbus8 target 0 lun 0 (pass16,da16)
[root@nas] ~# gpart show
=> 34 3907029101 da0 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 da1 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 da2 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 da3 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 da4 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 da5 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 da6 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 da7 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 da8 GPT (1.8T)
34 94 - free - (47k)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

Segmentation fault (core dumped)
[root@nas] ~#

 

[solarisguy]

It looks like your da9 has failed.

Use View Disks (an option in Storage) to identify disks by serial number and replace da9 (the missing one!) with a working hard drive.

 

[Adi]

Thanks for the quick reply. If I go to Storage -> View Disks, the da9 shows up. If i select it and press Edit, i can't see its serial number.. check out the screen shots.

Any idea ?

 

[solarisguy]

Then you have to do
smartctl -a /dev/DISK
15 times, with DISK taking values from da0 to da8, and then from da10 to da15.

smartctl -a /dev/da9 would hang, so do not execute such a command.

 

[gpsguy]

Or, you could try @Bidule0hm excellent script "Display drives identification infos (Device, GPTID, Serial)". Exclude da9 from the list of drives.

https://forums.freenas.org/index.ph...d-identification-and-backup-the-config.27365/