After upgrading from FreeNAS-9.10.2-U6 (561f0d7a1) to FreeNAS-11.1-U7, I've lost a pool, with 4 disks being marked as UNAVAIL (or more precisely 4 out of 6 disks in a Z1 vdev).
In the GUI, I get this message:
Before (after expanding the pool in January 2019):
After
I would be surprised if the HBA (or channel A of the HBA) or 4 disks failed simultaneously. (Although I know it's a possibility.)
Here's what the disks look like now individually:
Here's the part of the boot log related to HDD:
I haven't issued any zfs commands that write or update config and have only rebooted the system a few times to have a look at the EFI settings and watch the boot.
I am ok in terms of backup -- I snapshot/replicate this server's data to a twin, but I am concerned that an upgrade seemingly tanked the pool.
Questions:
In the GUI, I get this message:
Before (after expanding the pool in January 2019):
Code:
[root@freenas02] /mnt/vol01# zpool list -v
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
freenas-boot 7.44G 5.70G 1.74G - - 76% 1.00x ONLINE -
mirror 7.44G 5.70G 1.74G - - 76%
gptid/bd74e4fb-4470-11e5-bb9f-0014fd1702be - - - - - -
gptid/1d862695-c6e5-11e4-b757-0014fd1702be - - - - - -
vol01 32.5T 15.2T 17.3T - 19% 46% 1.00x ONLINE /mnt
raidz1 16.2T 15.2T 1.04T - 39% 93%
gptid/bd56e573-8b83-11e6-9a21-d05099c10963 - - - - - -
gptid/d65641e2-b536-11e6-9d70-d05099c10963 - - - - - -
gptid/bee5773e-8b83-11e6-9a21-d05099c10963 - - - - - -
gptid/bf9a5e0b-8b83-11e6-9a21-d05099c10963 - - - - - -
gptid/c0540768-8b83-11e6-9a21-d05099c10963 - - - - - -
gptid/c110d0a2-8b83-11e6-9a21-d05099c10963 - - - - - -
raidz1 16.2T 17.0M 16.2T - 0% 0%
da0 - - - - - -
da1 - - - - - -
da2 - - - - - -
da3 - - - - - -
da4 - - - - - -
da5 - - - - - -After
Code:
root@freenas02:~ # zpool import -V
pool: vol01
id: 10112117251247527724
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://illumos.org/msg/ZFS-8000-3C
config:
vol01 UNAVAIL insufficient replicas
raidz1-0 ONLINE
gptid/bd56e573-8b83-11e6-9a21-d05099c10963 ONLINE
gptid/d65641e2-b536-11e6-9d70-d05099c10963 ONLINE
gptid/bee5773e-8b83-11e6-9a21-d05099c10963 ONLINE
gptid/bf9a5e0b-8b83-11e6-9a21-d05099c10963 ONLINE
gptid/c0540768-8b83-11e6-9a21-d05099c10963 ONLINE
gptid/c110d0a2-8b83-11e6-9a21-d05099c10963 ONLINE
raidz1-1 UNAVAIL insufficient replicas
da0 ONLINE
da1 ONLINE
16339273880762305465 UNAVAIL cannot open
10436155621961562286 UNAVAIL cannot open
5238054744404005019 UNAVAIL cannot open
3466877252226870208 UNAVAIL cannot openI would be surprised if the HBA (or channel A of the HBA) or 4 disks failed simultaneously. (Although I know it's a possibility.)
Here's what the disks look like now individually:
Code:
root@freenas02:~ # camcontrol devlist -v
scbus0 on mps0 bus 0:
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 4 lun 0 (pass0,da0)
<ATA WDC WD30EFRX-68E 0A82> at scbus0 target 5 lun 0 (pass1,da1)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 6 lun 0 (pass2,da2)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 8 lun 0 (pass3,da3)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 9 lun 0 (pass4,da4)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 10 lun 0 (pass5,da5)
<> at scbus0 target -1 lun ffffffff ()
scbus1 on ahcich0 bus 0:
<WDC WD30EFRX-68AX9N0 80.00A80> at scbus1 target 0 lun 0 (pass6,ada0)
<> at scbus1 target -1 lun ffffffff ()
scbus2 on ahcich1 bus 0:
<WDC WD30EFRX-68AX9N0 80.00A80> at scbus2 target 0 lun 0 (pass7,ada1)
<> at scbus2 target -1 lun ffffffff ()
scbus3 on ahcich2 bus 0:
<WDC WD30EFRX-68AX9N0 80.00A80> at scbus3 target 0 lun 0 (pass8,ada2)
<> at scbus3 target -1 lun ffffffff ()
scbus4 on ahcich3 bus 0:
<WDC WD30EFRX-68EUZN0 82.00A82> at scbus4 target 0 lun 0 (pass9,ada3)
<> at scbus4 target -1 lun ffffffff ()
scbus5 on ahcich4 bus 0:
<WDC WD30EFRX-68EUZN0 82.00A82> at scbus5 target 0 lun 0 (pass10,ada4)
<> at scbus5 target -1 lun ffffffff ()
scbus6 on ahcich5 bus 0:
<WDC WD30EFRX-68EUZN0 82.00A82> at scbus6 target 0 lun 0 (pass11,ada5)
<> at scbus6 target -1 lun ffffffff ()
scbus7 on camsim0 bus 0:
<> at scbus7 target -1 lun ffffffff ()
scbus8 on umass-sim0 bus 0:
<SanDisk Cruzer Fit 1.26> at scbus8 target 0 lun 0 (da6,pass12)
scbus-1 on xpt0 bus 0:
<> at scbus-1 target -1 lun ffffffff (xpt0)
root@freenas02:~ # gpart show
=> 34 5860533101 da2 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 da3 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 da4 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 da5 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 15633341 da6 GPT (7.5G)
34 1024 1 freebsd-boot (512K)
1058 6 - free - (3.0K)
1064 15632304 2 freebsd-zfs (7.5G)
15633368 7 - free - (3.5K)
=> 34 15633341 da7 GPT (7.5G)
34 1024 1 freebsd-boot (512K)
1058 6 - free - (3.0K)
1064 15632304 2 freebsd-zfs (7.5G)
15633368 7 - free - (3.5K)
=> 34 5860533101 ada0 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 ada1 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 ada2 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 ada3 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 ada4 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 ada5 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 da0 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
=> 34 5860533101 da1 GPT (2.7T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338696 2 freebsd-zfs (2.7T)
5860533128 7 - free - (3.5K)
root@freenas02:~ # glabel status
Name Status Components
gptid/b028e667-f483-11e4-a2e2-ac220b4f4343 N/A da2p2
gptid/aec2bf48-f483-11e4-a2e2-ac220b4f4343 N/A da3p2
gptid/afc5b4b8-f483-11e4-a2e2-ac220b4f4343 N/A da4p2
gptid/be17ff68-8b83-11e6-9a21-d05099c10963 N/A da5p2
gptid/bd45140b-4470-11e5-bb9f-0014fd1702be N/A da6p1
gptid/bd74e4fb-4470-11e5-bb9f-0014fd1702be N/A da6p2
gptid/1d7804a0-c6e5-11e4-b757-0014fd1702be N/A da7p1
gptid/1d862695-c6e5-11e4-b757-0014fd1702be N/A da7p2
gptid/bf9a5e0b-8b83-11e6-9a21-d05099c10963 N/A ada0p2
gptid/c0540768-8b83-11e6-9a21-d05099c10963 N/A ada1p2
gptid/bee5773e-8b83-11e6-9a21-d05099c10963 N/A ada2p2
gptid/c110d0a2-8b83-11e6-9a21-d05099c10963 N/A ada3p2
gptid/bd56e573-8b83-11e6-9a21-d05099c10963 N/A ada4p2
gptid/d65641e2-b536-11e6-9d70-d05099c10963 N/A ada5p2
gptid/b07c7234-f483-11e4-a2e2-ac220b4f4343 N/A da0p1
gptid/b088f945-f483-11e4-a2e2-ac220b4f4343 N/A da0p2
gptid/b121c1c9-f483-11e4-a2e2-ac220b4f4343 N/A da1p1
gptid/b12d6a01-f483-11e4-a2e2-ac220b4f4343 N/A da1p2
Here's the part of the boot log related to HDD:
Code:
… Aug 17 13:46:54 freenas02 ahci0: <Intel Lynx Point AHCI SATA controller> port 0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0xf7c15000-0xf7c157ff irq 19 at device 31.2 on pci0 Aug 17 13:46:54 freenas02 ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported Aug 17 13:46:54 freenas02 ahcich0: <AHCI channel> at channel 0 on ahci0 Aug 17 13:46:54 freenas02 ahcich1: <AHCI channel> at channel 1 on ahci0 Aug 17 13:46:54 freenas02 ahcich2: <AHCI channel> at channel 2 on ahci0 Aug 17 13:46:54 freenas02 ahcich3: <AHCI channel> at channel 3 on ahci0 Aug 17 13:46:54 freenas02 ahcich4: <AHCI channel> at channel 4 on ahci0 Aug 17 13:46:54 freenas02 ahcich5: <AHCI channel> at channel 5 on ahci0 … Aug 17 13:46:54 freenas02 mps0: SAS Address for SATA device = d2624739dcc8c074 Aug 17 13:46:54 freenas02 mps0: SAS Address for SATA device = d267514bf8ccb972 Aug 17 13:46:54 freenas02 mps0: SAS Address for SATA device = d2624738ddc8c074 Aug 17 13:46:54 freenas02 mps0: SAS Address for SATA device = d2624738d9c3bd76 Aug 17 13:46:54 freenas02 mps0: SAS Address for SATA device = d2624734dfcabe71 Aug 17 13:46:54 freenas02 mps0: SAS Address for SATA device = d2624737dbc8b86d Aug 17 13:46:54 freenas02 mps0: SAS Address from SATA device = d2624739dcc8c074 Aug 17 13:46:54 freenas02 mps0: SAS Address from SATA device = d267514bf8ccb972 Aug 17 13:46:54 freenas02 mps0: SAS Address from SATA device = d2624738ddc8c074 Aug 17 13:46:54 freenas02 mps0: SAS Address from SATA device = d2624738d9c3bd76 Aug 17 13:46:54 freenas02 mps0: SAS Address from SATA device = d2624734dfcabe71 Aug 17 13:46:54 freenas02 mps0: SAS Address from SATA device = d2624737dbc8b86d … Aug 17 13:46:54 freenas02 ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 Aug 17 13:46:54 freenas02 da0 at mps0 bus 0 scbus0 target 4 lun 0 Aug 17 13:46:54 freenas02 ada0: <WDC WD30EFRX-68AX9N0 80.00A80> ACS-2 ATA SATA 3.x device Aug 17 13:46:54 freenas02 ada0: Serial Number WD-WCC1T0454382 Aug 17 13:46:54 freenas02 da0: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SPC-4 SCSI device Aug 17 13:46:54 freenas02 ada0: 600.000MB/s transfersda0: Serial Number WD-WMC4N0755697 Aug 17 13:46:54 freenas02 da0: 600.000MB/s transfers Aug 17 13:46:54 freenas02 da0: Command Queueing enabled Aug 17 13:46:54 freenas02 da0: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 da0: quirks=0x8<4K> Aug 17 13:46:54 freenas02 da1 at mps0 bus 0 scbus0 target 5 lun 0 Aug 17 13:46:54 freenas02 da1: <ATA WDC WD30EFRX-68E 0A82> Fixed Direct Access SPC-4 SCSI device Aug 17 13:46:54 freenas02 da1: Serial Number WD-WCC4N5AHPD25 Aug 17 13:46:54 freenas02 da1: 600.000MB/s transfers Aug 17 13:46:54 freenas02 da1: Command Queueing enabled Aug 17 13:46:54 freenas02 da1: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 da1: quirks=0x8<4K> Aug 17 13:46:54 freenas02 da2 at mps0 bus 0 scbus0 target 6 lun 0 Aug 17 13:46:54 freenas02 (SATA 3.x, UDMA6, PIO 8192bytes) Aug 17 13:46:54 freenas02 ada0: Command Queueing enabled Aug 17 13:46:54 freenas02 da2: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SPC-4 SCSI device Aug 17 13:46:54 freenas02 ada0: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 ada0: quirks=0x1<4K> Aug 17 13:46:54 freenas02 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 Aug 17 13:46:54 freenas02 da2: Serial Number WD-WMC4N0764697 Aug 17 13:46:54 freenas02 da2: 600.000MB/s transfersada1: <WDC WD30EFRX-68AX9N0 80.00A80> ACS-2 ATA SATA 3.x device Aug 17 13:46:54 freenas02 ada1: Serial Number WD-WCC1T0418229 Aug 17 13:46:54 freenas02 ada1: 600.000MB/s transfers Aug 17 13:46:54 freenas02 da2: Command Queueing enabled Aug 17 13:46:54 freenas02 da2: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 (SATA 3.x, da2: quirks=0x8<4K> Aug 17 13:46:54 freenas02 da3 at mps0 bus 0 scbus0 target 8 lun 0 Aug 17 13:46:54 freenas02 da3: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SPC-4 SCSI device Aug 17 13:46:54 freenas02 da3: Serial Number WD-WMC4N0717874 Aug 17 13:46:54 freenas02 da3: 600.000MB/s transfers Aug 17 13:46:54 freenas02 da3: Command Queueing enabled Aug 17 13:46:54 freenas02 UDMA6, PIO 8192bytes) Aug 17 13:46:54 freenas02 ada1: Command Queueing enabled Aug 17 13:46:54 freenas02 ada1: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 da3: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 da3: quirks=0x8<4K> Aug 17 13:46:54 freenas02 da4 at mps0 bus 0 scbus0 target 9 lun 0 Aug 17 13:46:54 freenas02 da4: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SPC-4 SCSI device Aug 17 13:46:54 freenas02 da4: Serial Number WD-WMC4N0751169 Aug 17 13:46:54 freenas02 da4: 600.000MB/s transfers Aug 17 13:46:54 freenas02 da4: Command Queueing enabled Aug 17 13:46:54 freenas02 da4: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 da4: quirks=0x8<4K> Aug 17 13:46:54 freenas02 da5 at mps0 bus 0 scbus0 target 10 lun 0 Aug 17 13:46:54 freenas02 da5: <ATA WDC WD30EFRX-68E 0A80> Fixed Direct Access SPC-4 SCSI device Aug 17 13:46:54 freenas02 ada1: quirks=0x1<4K> Aug 17 13:46:54 freenas02 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 Aug 17 13:46:54 freenas02 da5: Serial Number WD-WMC4N0743610 Aug 17 13:46:54 freenas02 ada2: <WDC WD30EFRX-68AX9N0 80.00A80> ACS-2 ATA SATA 3.x device Aug 17 13:46:54 freenas02 da5: 600.000MB/s transfers Aug 17 13:46:54 freenas02 da5: Command Queueing enabled Aug 17 13:46:54 freenas02 ada2: Serial Number WD-WCC1T0298346 Aug 17 13:46:54 freenas02 ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) Aug 17 13:46:54 freenas02 ada2: Command Queueing enabled Aug 17 13:46:54 freenas02 ada2: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 da5: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 da5: quirks=0x8<4K> Aug 17 13:46:54 freenas02 ada2: quirks=0x1<4K> Aug 17 13:46:54 freenas02 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 Aug 17 13:46:54 freenas02 ada3: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device Aug 17 13:46:54 freenas02 ada3: Serial Number WD-WCC4N2TYF2UN Aug 17 13:46:54 freenas02 ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) Aug 17 13:46:54 freenas02 ada3: Command Queueing enabled Aug 17 13:46:54 freenas02 ada3: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 ada3: quirks=0x1<4K> Aug 17 13:46:54 freenas02 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 Aug 17 13:46:54 freenas02 ada4: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device Aug 17 13:46:54 freenas02 ada4: Serial Number WD-WCC4N7FATRR1 Aug 17 13:46:54 freenas02 ada4: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) Aug 17 13:46:54 freenas02 ada4: Command Queueing enabled Aug 17 13:46:54 freenas02 ada4: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 ada4: quirks=0x1<4K> Aug 17 13:46:54 freenas02 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 Aug 17 13:46:54 freenas02 ada5: <WDC WD30EFRX-68EUZN0 82.00A82> ACS-2 ATA SATA 3.x device Aug 17 13:46:54 freenas02 ada5: Serial Number WD-WCC4N4FCUHHY Aug 17 13:46:54 freenas02 ada5: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) Aug 17 13:46:54 freenas02 ada5: Command Queueing enabled Aug 17 13:46:54 freenas02 ada5: 2861588MB (5860533168 512 byte sectors) Aug 17 13:46:54 freenas02 ada5: quirks=0x1<4K> Aug 17 13:46:54 freenas02 Trying to mount root from zfs:freenas-boot/ROOT/11.1-U7 []... … Aug 17 14:49:46 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=3671607037181642970 Aug 17 14:49:46 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=9652509477325859936 Aug 17 14:49:47 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=14436258189133544651 Aug 17 14:49:47 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=13407339990873068916 Aug 17 14:49:47 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=11357776941345595226 Aug 17 14:49:47 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=17955406522460418432 Aug 17 14:49:47 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=3535278873160926383 Aug 17 14:49:47 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=15861408793602997199 Aug 17 14:49:48 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=16339273880762305465 Aug 17 14:49:48 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=10436155621961562286 Aug 17 14:49:48 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=5238054744404005019 Aug 17 14:49:48 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=3466877252226870208 … Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=3671607037181642970 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=9652509477325859936 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=14436258189133544651 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=13407339990873068916 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=11357776941345595226 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=17955406522460418432 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=3535278873160926383 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=15861408793602997199 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=16339273880762305465 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=10436155621961562286 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=5238054744404005019 Aug 17 14:51:18 freenas02 ZFS: vdev state changed, pool_guid=10112117251247527724 vdev_guid=3466877252226870208 … Aug 17 14:53:45 freenas02 ZFS: vdev state changed, pool_guid=3018436033628833179 vdev_guid=13331606250362451550 Aug 17 14:53:46 freenas02 ZFS: vdev state changed, pool_guid=3018436033628833179 vdev_guid=8155704046927289062 …
I haven't issued any zfs commands that write or update config and have only rebooted the system a few times to have a look at the EFI settings and watch the boot.
I am ok in terms of backup -- I snapshot/replicate this server's data to a twin, but I am concerned that an upgrade seemingly tanked the pool.
Questions:
- Can I recover these 4 disks and import the pool? (tweaking a few settings seems faster than destroying the remnants of the pool and then zfs sending 15 TB+ over the wire from the backup server) If so, what are the next steps?
- Are there any known issues with the 11.x upgrades and a pool that has been extended this way?
- Should I expect this behaviour when I upgrade the backup server as well?
