ZPOOL UNKNOWN after reboot

[W3rt3lka]

Hello,

After FreeNAS reboot (version FreeNAS-11.2-U2.1) ZPOOL did not came up and now showing UNKNOWN.

Code:

root@ch-freenas-n1:~ # zpool import
   pool: Data
     id: 3635807659537760243
  state: FAULTED
status: The pool was last accessed by another system.
action: The pool cannot be imported due to damaged devices or data.
        The pool may be active on another system, but can be imported using
        the '-f' flag.
   see: http://illumos.org/msg/ZFS-8000-EY
config:

        Data                                            FAULTED  corrupted data
          raidz3-0                                      ONLINE
            gptid/50b684f8-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/58212a15-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/5f783e70-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/66de9983-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/6e4c8f92-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/75d3c417-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/7d63a3f6-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/84d34b98-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/8c68a479-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/93e50361-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/9b72c864-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/a366c588-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/ab4daa4e-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/b3717e69-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/bb92f21a-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/c34ac276-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/cb390fe2-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/d2db5a02-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/da8911ab-710b-11e9-bb4e-002590fcf85e  ONLINE
        logs
          mirror-1                                      ONLINE
            gptid/efd69cb9-710b-11e9-bb4e-002590fcf85e  ONLINE
            gptid/f6d16b17-710b-11e9-bb4e-002590fcf85e  ONLINE

Code:

root@ch-freenas-n1:~ # zpool import -f Data
cannot import 'Data': I/O error
        Destroy and re-create the pool from
        a backup source.

I tried zpool import -F I tried zpool import -fFX command but system kernel crash and system reboots.

After i done some research i found three disks of 24 file system not able to import partition information. All disks are in healthy state. I need rebuild ZFS. Please help!!

 

[Chris Moore]

What hardware are you using.

 

[W3rt3lka]

SuperMicro chasis with two controlers (X9DBS-F(-2U) ) with local hard drive bay 19x 1 TB 7.2k HDD and 4x 400 GB SSD
Direct access to disks, no hardware RAID

 

[joeschmuck]

Can you describe your "Data" pool, such as how many drives, RAIDZ/Mirror/Etc... Have you run a SMART Short Test and Long Test on them? The Short test could help determine if you have a data cable issue (UDMA_CRC_Error_Count) or if you have other damage that has been recorded whereas the Long Test will surface test the entire surface area (non-destructive) and if it finds error it will report those as well.

All disks are in healthy state.

How was that determined?

 

[danb35]

Can you describe your "Data" pool, such as how many drives, RAIDZ/Mirror/Etc

That would appear to be answered in the OP by the output of zpool import. It's a 19-disk-wide RAIDZ3 vdev with mirrored SLOG devices. What on earth possessed OP to configure his pool that way is a separate question.

 

[sretalla]

i found three disks of 24 file system not able to import partition information

the output of zpool import. It's a 19-disk-wide RAIDZ3 vdev with mirrored SLOG devices

Something not adding up here... 24 disks, 3 not importing partition information (not sure what that means). 19 + 2 in the zpool import/status if the 3 "broken disks" are part of SLOG/L2ARC, maybe that's some part of the issue, but I would expect to have seen something in the output indicating something is wrong/missing.

 

[Chris Moore]

After FreeNAS reboot (version FreeNAS-11.2-U2.1) ZPOOL did not came up and now showing UNKNOWN.

Did you recently upgrade the operating system version?

SuperMicro chasis with two controlers (X9DBS-F(-2U) ) with local hard drive bay 19x 1 TB 7.2k HDD and 4x 400 GB SSD
Direct access to disks, no hardware RAID

This model system board... Which you appear to be calling a controller... I looked at details of that.
https://www.supermicro.com/products/motherboard/Xeon/C600/X9DBS-F-2U.cfm
https://www.supermicro.com/products/system/2U/2027/SSG-2027B-DE2R24L.cfm
It is a node board in a multi-node chassis. What controls the connection between the system board and the chassis disks? What is the exact details of the disk controller configuration?

 

[W3rt3lka]

Something not adding up here... 24 disks, 3 not importing partition information (not sure what that means). 19 + 2 in the zpool import/status if the 3 "broken disks" are part of SLOG/L2ARC, maybe that's some part of the issue, but I would expect to have seen something in the output indicating something is wrong/missing.

Data - 19 7.2k HDD
SLOG- 2 SSD's
L2RC - 2 SSD's
1- HDD for Hotspare
Total: 24 disks

When FreeNAS boots up and system trying import Data pool it does import disk by disk and i see on disk da5, da6, da7 operation fails. Adding screenshot attachment.

 

[W3rt3lka]

Did you recently upgrade the operating system version?

This model system board... Which you appear to be calling a controller... I looked at details of that.
https://www.supermicro.com/products/motherboard/Xeon/C600/X9DBS-F-2U.cfm
https://www.supermicro.com/products/system/2U/2027/SSG-2027B-DE2R24L.cfm
It is a node board in a multi-node chassis. What controls the connection between the system board and the chassis disks? What is the exact details of the disk controller configuration?

One chasis with two controllers. No RAID configuration, FreeNAS sees disks directly.

 

[joeschmuck]

That would appear to be answered in the OP by the output of zpool import. It's a 19-disk-wide RAIDZ3 vdev with mirrored SLOG devices. What on earth possessed OP to configure his pool that way is a separate question.

Guess I didn't see the scroll bar, my bad.

 

[danb35]

One chasis with two controllers.

They aren't controllers, they're motherboards. Feel free to answer questions with complete sentences (or even multiple sentences).

Data - 19 7.2k HDD
SLOG- 2 SSD's
L2RC - 2 SSD's
1- HDD for Hotspare

The spare and the L2ARC aren't showing at all.

Can you post the output of zpool import in code tags? Indenting matters, and your OP omits that. (I hadn't seen the code formatting before, but now it's there)