HDD falling out of pool?

tfran1990 · Apr 15, 2020

I have only seen this happen one time so far, to me it seems like the disk has fell out of the pool.
i also see a CAM error.

should i replace the drive?

Code:

Apr 14 01:14:01 freenas da5 at mps0 bus 0 scbus0 target 9 lun 0
Apr 14 01:14:01 freenas da5: <ATA WDC WD20EZAZ-00G 0A80> s/n WD-WXP1AC82RNH9 detached
Apr 14 01:14:01 freenas     (da5:mps0:0:9:0): ATA COMMAND PASS THROUGH(16). CDB: 85 0d 06 00 01 00 01 00 00 00 00 00 00 40 06 00 length 512 SMID 408 terminated ioc 804b loginfo 31110d00 scsi 0 state c xfer 0
Apr 14 01:14:01 freenas     (da5:mps0:0:9:0): WRITE(10). CDB: 2a 00 41 32 da 28 00 00 08 00 length 4096 SMID 815 terminated ioc 804b loginfo 31110d00 scs(da5:mps0:0:9:0): ATA COMMAND PASS THROUGH(16). CDB: 85 0d 06 00 01 00 01 00 00 00 00 00 00 40 06 00
Apr 14 01:14:01 freenas i 0 state c xfer 0
Apr 14 01:14:01 freenas (da5:mps0:0:9:0): CAM status: CCB request completed with an error
Apr 14 01:14:01 freenas (da5:mps0:0:9:0): Error 5, Periph was invalidated
Apr 14 01:14:01 freenas mps0: Unfreezing devq for target ID 9
Apr 14 01:14:01 freenas (da5:mps0:0:9:0): WRITE(10). CDB: 2a 00 41 32 da 28 00 00 08 00
Apr 14 01:14:01 freenas (da5:mps0:0:9:0): CAM status: CCB request completed with an error
Apr 14 01:14:01 freenas (da5:mps0:0:9:0): Error 5, Periph was invalidated
Apr 14 01:14:01 freenas GEOM_MIRROR: Device swap0: provider da5p1 disconnected.
Apr 14 01:14:02 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=17276462857189123742
Apr 14 01:14:02 freenas (da5:mps0:0:9:0): Periph destroyed
Apr 14 01:14:03 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=17276462857189123742
Apr 14 01:14:06 freenas mps0: SAS Address for SATA device = 456f3f37ecefbf75
Apr 14 01:14:06 freenas mps0: SAS Address from SATA device = 456f3f37ecefbf75
Apr 14 01:14:06 freenas da5 at mps0 bus 0 scbus0 target 9 lun 0
Apr 14 01:14:06 freenas da5: <ATA WDC WD20EZAZ-00G 0A80> Fixed Direct Access SPC-4 SCSI device
Apr 14 01:14:06 freenas da5: Serial Number WD-WXP1AC82RNH9
Apr 14 01:14:06 freenas da5: 600.000MB/s transfers
Apr 14 01:14:06 freenas da5: Command Queueing enabled
Apr 14 01:14:06 freenas da5: 1907729MB (3907029168 512 byte sectors)
Apr 14 01:14:06 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=14954804824692267039
Apr 14 01:14:06 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=8134557650828204655
Apr 14 01:14:06 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=17276462857189123742
Apr 14 01:14:06 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=4624288917331818763
Apr 14 01:14:06 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=3004686278740164286
Apr 14 01:14:15 freenas mps0: mpssas_prepare_remove: Sending reset for target ID 9
Apr 14 01:14:15 freenas da5 at mps0 bus 0 scbus0 target 9 lun 0
Apr 14 01:14:15 freenas da5: <ATA WDC WD20EZAZ-00G 0A80> s/n WD-WXP1AC82RNH9 detached
Apr 14 01:14:16 freenas     (da5:mps0:0:9:0): WRITE(10). CDB: 2a 00 41 32 db 40 00 00 08 00 length 4096 SMID 834 terminated ioc 804b loginfo 31110d00 scsi 0 state c xfer 0
Apr 14 01:14:16 freenas (da5:mps0:0:9:0): WRITE(10). CDB: 2a 00 41 32 db 40 00 00 08 00
Apr 14 01:14:16 freenas (da5:mps0:0:9:0): CAM status: CCB request completed with an error
Apr 14 01:14:16 freenas mps0: (da5:mps0:0:9:0): Error 5, Periph was invalidated
Apr 14 01:14:16 freenas Unfreezing devq for target ID 9
Apr 14 01:14:16 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=17276462857189123742
Apr 14 01:14:16 freenas (da5:mps0:0:9:0): Periph destroyed
Apr 14 01:14:20 freenas mps0: SAS Address for SATA device = 456f3f37ecefbf75
Apr 14 01:14:20 freenas mps0: SAS Address from SATA device = 456f3f37ecefbf75
Apr 14 01:14:20 freenas da5 at mps0 bus 0 scbus0 target 9 lun 0
Apr 14 01:14:20 freenas da5: <ATA WDC WD20EZAZ-00G 0A80> Fixed Direct Access SPC-4 SCSI device
Apr 14 01:14:20 freenas da5: Serial Number WD-WXP1AC82RNH9
Apr 14 01:14:20 freenas da5: 600.000MB/s transfers
Apr 14 01:14:20 freenas da5: Command Queueing enabled
Apr 14 01:14:20 freenas da5: 1907729MB (3907029168 512 byte sectors)
Apr 14 01:15:10 freenas ZFS: vdev state changed, pool_guid=6363471716941292545 vdev_guid=17276462857189123742
Apr 14 01:17:09 freenas GEOM_ELI: Device mirror/swap0.eli destroyed.
Apr 14 01:17:09 freenas GEOM_MIRROR: Device swap0: provider destroyed.
Apr 14 01:17:09 freenas GEOM_MIRROR: Device swap0 destroyed.
Apr 14 01:17:09 freenas GEOM_MIRROR: Device mirror/swap0 launched (2/2).
Apr 14 01:17:09 freenas GEOM_ELI: Device mirror/swap0.eli created.
Apr 14 01:17:09 freenas GEOM_ELI: Encryption: AES-XTS 128
Apr 14 01:17:09 freenas GEOM_ELI:     Crypto: hardware

Code:

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   179   171   021    Pre-fail  Always       -       2041
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       14
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   093   093   000    Old_age   Always       -       5110
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       14
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       12
193 Load_Cycle_Count        0x0032   101   101   000    Old_age   Always       -       297616
194 Temperature_Celsius     0x0022   113   108   000    Old_age   Always       -       30
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%      5105         -
# 2  Short offline       Completed without error       00%      4868         -
# 3  Short offline       Completed without error       00%      4652         -
# 4  Extended offline    Completed without error       00%      4537         -
# 5  Short offline       Completed without error       00%      4365         -
# 6  Short offline       Completed without error       00%      4136         -
# 7  Short offline       Completed without error       00%      3968         -
# 8  Extended offline    Completed without error       00%      3859         -
# 9  Short offline       Completed without error       00%      3680         -
#10  Extended offline    Completed without error       00%      3123         -
#11  Extended offline    Completed without error       00%      2395         -
#12  Short offline       Completed without error       00%      1728         -
#13  Extended offline    Completed without error       00%      1679         -
#14  Extended offline    Completed without error       00%      1308         -
#15  Extended offline    Completed without error       00%       935         -
#16  Extended offline    Completed without error       00%       211         -
#17  Extended offline    Completed without error       00%        38         -
#18  Conveyance offline  Completed without error       00%         0         -
#19  Short offline       Completed without error       00%         0         -

HoneyBadger · Apr 15, 2020

That appears to be a lot of load cycles (297K) for a low number of power-on hours, it's probably been starting/stopping which has worn things out over time.

That's also an SMR drive which won't do you any favors either: https://www.ixsystems.com/community/threads/wd-sneaking-smr-into-red-disks.83972/ - this may be the cause of the failures to write.

Short answer: Yes, replace it.

tfran1990 · Apr 15, 2020

So once the drive connected back the write was complete, or do i need to scrub?

The drive still has a warranty, bad thing about it is ill get another SMR disk back

Important Announcement for the TrueNAS Community.

HDD falling out of pool?

tfran1990

Patron

HoneyBadger

actually does care

tfran1990

Patron

Similar threads