Recover

nightowl · Apr 16, 2019

This is a home built DIY unit that has worked flawlessly since 2015.

Hardware and Software...

Motherboard: ASRock AM1B-ITX
CPU: AMD Sempron 3850
RAM: 8 GB (two sticks) XPG V1.0 DDR3
HDD: Four (4) WD Red 3TB
All stuck in an old Dell case with a Dell PSU powering the system.
Boot disk: OS on SanDisk Cruzer Fit 8 GB USB flash drive
Using NIC on motherboard
No drive controllers (unless they are somehow built into the motherboard)
Latest build: FreeNAS build: FreeNAS-11.1-U7

Yesterday while the unit was idling, (I was out of the office and nothing would have been connecting to it) something goofed up. When I came back, I could not establish an NFS connection to my share.

When I logged into the GUI, there was a system alert: Critical: <Timestamp> - The volume <VolumeName> state is UNKNOWN
And then below that an yellow alert saying I had used 81% of my storage space.

I sent the device through a reboot. And it took a long time for it to reboot.

During the reboot process, I noticed the following in the output...

Code:

Importing <a big long number>

cannot import 'VolumeName' : no such pool or dataset

Destroy and recreate the pool from a backup source

cannot import 'VolumeName' : one or more devices is currectly unavailable

Importing VolumeName

cannot import 'VolumeName': one or more devices is currently unavailable

cannot open 'VolumeName': no such pool

During the boot process, all four drives are showing up.

I have spent the last day searching this forum and on-line but do not have a clear picture of figuring out how to figure out what is broken and then fix it.

Thanks for your help.

Chris Moore · Apr 16, 2019

How were the drives organized into a storage pool? Mirrors, RAIDz2? What kind of monitoring scripts did you have running to keep an eye on the system health? When did you check it last?
The error you described sounds like you have a drive that has failed making the pool unavailable.

nightowl · Apr 16, 2019

Thanks Chris.

I am still reading up on how to check drives.

I do not remember what configuration I had the volume set up with and cannot figure out how to find it out since the volume is not loading. If there is there a way to dig it out of a configuration file, I have one of those to guard against boot disk failure.

Before setting this up, I did some reading up on the configurations and choosing the one that offered the greatest data security in case of drive failure. I believe, believe, is was "striped vdevs", but am not 100% sure. I had 12 TB of drive space and after configuration had somewhere between 6 and 8 TB of drive space.

Beyond that, it was a default setup. No scripts were set up other than a weekly scrub on the boot disk and a daily snapshot of the volume. Other than that I would log in every week or two to check to see whether anything showed need of attention and to force the system to check for updates.

I am still learning about this so feel free to let me know where I am missing it.

nightowl · Apr 16, 2019

SMART Data is not showing any errors on any of the four drives...

Code:

root@freenas:~ # smartctl -l error /dev/ada0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Error Log Version: 1
No Errors Logged

root@freenas:~ # smartctl -l error /dev/ada1
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Error Log Version: 1
No Errors Logged

root@freenas:~ # smartctl -l error /dev/ada2
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Error Log Version: 1
No Errors Logged

root@freenas:~ # smartctl -l error /dev/ada3
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Error Log Version: 1
No Errors Logged

danb35 · Apr 16, 2019

nightowl said:
cannot figure out how to find it out since the volume is not loading.

What's the output of zpool import?

nightowl · Apr 16, 2019

@danb35 - Here is the output...

Code:

root@freenas:~ # zpool import
   pool: VolumeName
     id: 5057694033267979932
  state: UNAVAIL
 status: One or more devices are missing from the system.
 action: The pool cannot be imported. Attach the missing
    devices and try again.
   see: http://illumos.org/msg/ZFS-8000-6X
 config:

    VolumeName                                     UNAVAIL  missing device
      mirror-1                                          ONLINE
        gptid/6b64892f-1140-11e6-980e-d0509913e8a9.eli  ONLINE
        gptid/6c49e955-1140-11e6-980e-d0509913e8a9.eli  ONLINE

    Additional devices are known to be part of this pool, though their
    exact configuration cannot be determined.

SweetAndLow · Apr 16, 2019

Looks like you only have 2 disks, what happened to the other 2? go make sure they are plugged in and have power. Is this pool encrypted?

tfran1990 · Apr 16, 2019

were you using a HBA?
check connectors and cables.

Chris Moore · Apr 16, 2019

SweetAndLow said:
Is this pool encrypted?

Yes, you can tell that from the .eli on the end of the gptid.

Chris Moore · Apr 16, 2019

nightowl said:
SMART Data is not showing any errors on any of the four drives...

This is not how you check.

Github repository for FreeNAS scripts, including disk burnin
https://forums.freenas.org/index.ph...for-freenas-scripts-including-disk-burnin.28/

solnet-array-test (for drive / array speed) non destructive test
https://forums.freenas.org/index.php?resources/solnet-array-test.1/

Useful Commands
https://forums.freenas.org/index.php?threads/useful-commands.30314/#post-195192

Hard Drive Troubleshooting Guide (All Versions of FreeNAS)
https://forums.freenas.org/index.ph...bleshooting-guide-all-versions-of-freenas.17/

nightowl · Apr 16, 2019

@SweetAndLow - Thanks for asking.

I am not sure where the other two went. That is what I a trying to figure out. (Now that you all have asked the questions.)

@Chris Moore - You answered all the questions I was going to ask.

I will dig into this deeper tomorrow and report back as I learn more.

You all have a good evening.

Chris Moore · Apr 16, 2019

I would suggest that you post the output of smartctl –x /dev/ada# for each drive so we can have a look and see if there is something of importance because smartctl can't be trusted to give you the answer in that one line you posted.

The other thing that is potentially important is to look at the partition table of the drives to ensure they were all actually being used.
Using glabel status might help with that.

Chris Moore · Apr 16, 2019

nightowl said:
Before setting this up, I did some reading up on the configurations and choosing the one that offered the greatest data security in case of drive failure. I believe, believe, is was "striped vdevs", but am not 100% sure. I had 12 TB of drive space and after configuration had somewhere between 6 and 8 TB of drive space.

This plus the zpool import response makes me think you had a pair of striped mirror vdevs. I am not sure about this, but it looks likely.

nightowl · Apr 17, 2019

Chris Moore said:
I would suggest that you post the output of smartctl –x /dev/ada# for each drive so we can have a look...

The other thing that is potentially important is to look at the partition table of the drives to ensure they were all actually being used.
Using glabel status might help with that.

Coming up in the next five posts...

nightowl · Apr 17, 2019

root@freenas:~ # smartctl -x /dev/ada0

Code:

smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N6TYTX45
LU WWN Device Id: 5 0014ee 20af897c8
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 17 12:08:40 2019 -03
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Unavailable
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, NOT FROZEN [SEC1]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever
                    been run.
Total time to complete Offline
data collection:         (39840) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      ( 399) minutes.
Conveyance self-test routine
recommended polling time:      (   5) minutes.
SCT capabilities:            (0x703d)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    0
  3 Spin_Up_Time            POS--K   178   177   021    -    6066
  4 Start_Stop_Count        -O--CK   100   100   000    -    141
  5 Reallocated_Sector_Ct   PO--CK   200   200   140    -    0
  7 Seek_Error_Rate         -OSR-K   200   200   000    -    0
  9 Power_On_Hours          -O--CK   071   071   000    -    21553
10 Spin_Retry_Count        -O--CK   100   100   000    -    0
11 Calibration_Retry_Count -O--CK   100   100   000    -    0
12 Power_Cycle_Count       -O--CK   100   100   000    -    141
192 Power-Off_Retract_Count -O--CK   200   200   000    -    69
193 Load_Cycle_Count        -O--CK   200   200   000    -    384
194 Temperature_Celsius     -O---K   122   103   000    -    28
196 Reallocated_Event_Count -O--CK   200   200   000    -    0
197 Current_Pending_Sector  -O--CK   200   200   000    -    0
198 Offline_Uncorrectable   ----CK   100   253   000    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   200   000    -    0
200 Multi_Zone_Error_Rate   ---R--   100   253   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      6  Ext. Comprehensive SMART error log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa0-0xa7  GPL,SL  VS      16  Device vendor specific log
0xa8-0xb7  GPL,SL  VS       1  Device vendor specific log
0xbd       GPL,SL  VS       1  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL     VS      93  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (6 sectors)
Device Error Count: 1
    CR     = Command Register
    FEATR  = Features Register
    COUNT  = Count (was: Sector Count) Register
    LBA_48 = Upper bytes of LBA High/Mid/Low Registers ]  ATA-8
    LH     = LBA High (was: Cylinder High) Register    ]   LBA
    LM     = LBA Mid (was: Cylinder Low) Register      ] Register
    LL     = LBA Low (was: Sector Number) Register     ]
    DV     = Device (was: Device/Head) Register
    DC     = Device Control Register
    ER     = Error register
    ST     = Status register
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 1 [0] occurred at disk power-on lifetime: 14122 hours (588 days + 10 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  10 -- 51 00 00 00 01 4e 40 88 60 40 00  Error: IDNF at LBA = 0x14e408860 = 5607819360

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  61 00 08 00 60 00 01 4e 40 88 60 40 08     00:21:42.613  WRITE FPDMA QUEUED
  ea 00 00 00 00 00 00 00 00 00 00 40 08     00:21:37.733  FLUSH CACHE EXT
  61 00 08 00 50 00 01 5d 50 a2 70 40 08     00:21:37.733  WRITE FPDMA QUEUED
  61 00 08 00 48 00 01 5d 50 a0 70 40 08     00:21:37.733  WRITE FPDMA QUEUED
  61 00 08 00 40 00 00 00 40 04 70 40 08     00:21:37.733  WRITE FPDMA QUEUED

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       258 (0x0102)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    28 Celsius
Power Cycle Min/Max Temperature:     25/28 Celsius
Lifetime    Min/Max Temperature:      2/47 Celsius
Under/Over Temperature Limit Count:   0/0
Vendor specific:
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -41/85 Celsius
Temperature History Size (Index):    478 (348)

Index    Estimated Time   Temperature Celsius
349    2019-04-17 04:11    36  *****************
...    ..( 61 skipped).    ..  *****************
411    2019-04-17 05:13    36  *****************
412    2019-04-17 05:14    35  ****************
...    ..( 19 skipped).    ..  ****************
432    2019-04-17 05:34    35  ****************
433    2019-04-17 05:35    36  *****************
434    2019-04-17 05:36    36  *****************
435    2019-04-17 05:37    36  *****************
436    2019-04-17 05:38    35  ****************
...    ..( 43 skipped).    ..  ****************
   2    2019-04-17 06:22    35  ****************
   3    2019-04-17 06:23    34  ***************
   4    2019-04-17 06:24     ?  -
   5    2019-04-17 06:25    35  ****************
...    ..( 25 skipped).    ..  ****************
  31    2019-04-17 06:51    35  ****************
  32    2019-04-17 06:52    34  ***************
...    ..( 29 skipped).    ..  ***************
  62    2019-04-17 07:22    34  ***************
  63    2019-04-17 07:23    33  **************
  64    2019-04-17 07:24    33  **************
  65    2019-04-17 07:25    34  ***************
...    ..( 26 skipped).    ..  ***************
  92    2019-04-17 07:52    34  ***************
  93    2019-04-17 07:53    33  **************
...    ..( 33 skipped).    ..  **************
127    2019-04-17 08:27    33  **************
128    2019-04-17 08:28    34  ***************
...    ..(  7 skipped).    ..  ***************
136    2019-04-17 08:36    34  ***************
137    2019-04-17 08:37    33  **************
...    ..( 98 skipped).    ..  **************
236    2019-04-17 10:16    33  **************
237    2019-04-17 10:17    32  *************
...    ..(  6 skipped).    ..  *************
244    2019-04-17 10:24    32  *************
245    2019-04-17 10:25    33  **************
...    ..( 19 skipped).    ..  **************
265    2019-04-17 10:45    33  **************
266    2019-04-17 10:46    32  *************
...    ..( 42 skipped).    ..  *************
309    2019-04-17 11:29    32  *************
310    2019-04-17 11:30    33  **************
311    2019-04-17 11:31    32  *************
...    ..( 16 skipped).    ..  *************
328    2019-04-17 11:48    32  *************
329    2019-04-17 11:49     ?  -
330    2019-04-17 11:50    25  ******
331    2019-04-17 11:51    25  ******
332    2019-04-17 11:52    25  ******
333    2019-04-17 11:53    26  *******
334    2019-04-17 11:54    26  *******
335    2019-04-17 11:55    27  ********
336    2019-04-17 11:56    27  ********
337    2019-04-17 11:57    27  ********
338    2019-04-17 11:58    28  *********
...    ..(  3 skipped).    ..  *********
342    2019-04-17 12:02    28  *********
343    2019-04-17 12:03    36  *****************
...    ..(  4 skipped).    ..  *****************
348    2019-04-17 12:08    36  *****************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP/SMART Log 0x04) not supported

Pending Defects log (GP Log 0x0c) not supported

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x8000  4          802  Vendor specific

nightowl · Apr 17, 2019

root@freenas:~ # smartctl -x /dev/ada1

Code:

smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N0SN0YZP
LU WWN Device Id: 5 0014ee 2604dd258
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 17 12:11:29 2019 -03
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Unavailable
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, NOT FROZEN [SEC1]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever
                    been run.
Total time to complete Offline
data collection:         (39540) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      ( 397) minutes.
Conveyance self-test routine
recommended polling time:      (   5) minutes.
SCT capabilities:            (0x703d)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    50
  3 Spin_Up_Time            POS--K   180   177   021    -    5958
  4 Start_Stop_Count        -O--CK   100   100   000    -    146
  5 Reallocated_Sector_Ct   PO--CK   200   200   140    -    0
  7 Seek_Error_Rate         -OSR-K   200   200   000    -    0
  9 Power_On_Hours          -O--CK   071   071   000    -    21553
10 Spin_Retry_Count        -O--CK   100   100   000    -    0
11 Calibration_Retry_Count -O--CK   100   100   000    -    0
12 Power_Cycle_Count       -O--CK   100   100   000    -    146
192 Power-Off_Retract_Count -O--CK   200   200   000    -    74
193 Load_Cycle_Count        -O--CK   200   200   000    -    435
194 Temperature_Celsius     -O---K   122   102   000    -    28
196 Reallocated_Event_Count -O--CK   200   200   000    -    0
197 Current_Pending_Sector  -O--CK   200   200   000    -    0
198 Offline_Uncorrectable   ----CK   100   253   000    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   200   000    -    0
200 Multi_Zone_Error_Rate   ---R--   100   253   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      6  Ext. Comprehensive SMART error log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa0-0xa7  GPL,SL  VS      16  Device vendor specific log
0xa8-0xb7  GPL,SL  VS       1  Device vendor specific log
0xbd       GPL,SL  VS       1  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL     VS      93  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (6 sectors)
Device Error Count: 1
    CR     = Command Register
    FEATR  = Features Register
    COUNT  = Count (was: Sector Count) Register
    LBA_48 = Upper bytes of LBA High/Mid/Low Registers ]  ATA-8
    LH     = LBA High (was: Cylinder High) Register    ]   LBA
    LM     = LBA Mid (was: Cylinder Low) Register      ] Register
    LL     = LBA Low (was: Sector Number) Register     ]
    DV     = Device (was: Device/Head) Register
    DC     = Device Control Register
    ER     = Error register
    ST     = Status register
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 1 [0] occurred at disk power-on lifetime: 14122 hours (588 days + 10 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  10 -- 51 00 00 00 01 4e 40 88 60 40 00  Error: IDNF at LBA = 0x14e408860 = 5607819360

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  61 00 08 00 c8 00 01 4e 40 88 60 40 08     00:21:42.862  WRITE FPDMA QUEUED
  ea 00 00 00 00 00 00 00 00 00 00 40 08     00:21:37.982  FLUSH CACHE EXT
  61 00 08 00 b8 00 01 5d 50 a2 70 40 08     00:21:37.981  WRITE FPDMA QUEUED
  61 00 08 00 b0 00 01 5d 50 a0 70 40 08     00:21:37.981  WRITE FPDMA QUEUED
  61 00 08 00 a8 00 00 00 40 04 70 40 08     00:21:37.981  WRITE FPDMA QUEUED

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       258 (0x0102)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    28 Celsius
Power Cycle Min/Max Temperature:     25/28 Celsius
Lifetime    Min/Max Temperature:      2/48 Celsius
Under/Over Temperature Limit Count:   0/0
Vendor specific:
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -41/85 Celsius
Temperature History Size (Index):    478 (356)

Index    Estimated Time   Temperature Celsius
357    2019-04-17 04:14    36  *****************
...    ..(110 skipped).    ..  *****************
468    2019-04-17 06:05    36  *****************
469    2019-04-17 06:06    35  ****************
...    ..( 16 skipped).    ..  ****************
   8    2019-04-17 06:23    35  ****************
   9    2019-04-17 06:24     ?  -
  10    2019-04-17 06:25    35  ****************
...    ..(  4 skipped).    ..  ****************
  15    2019-04-17 06:30    35  ****************
  16    2019-04-17 06:31    36  *****************
...    ..(  6 skipped).    ..  *****************
  23    2019-04-17 06:38    36  *****************
  24    2019-04-17 06:39    35  ****************
...    ..( 21 skipped).    ..  ****************
  46    2019-04-17 07:01    35  ****************
  47    2019-04-17 07:02    34  ***************
...    ..( 78 skipped).    ..  ***************
126    2019-04-17 08:21    34  ***************
127    2019-04-17 08:22    33  **************
128    2019-04-17 08:23    33  **************
129    2019-04-17 08:24    33  **************
130    2019-04-17 08:25    34  ***************
...    ..( 35 skipped).    ..  ***************
166    2019-04-17 09:01    34  ***************
167    2019-04-17 09:02    33  **************
...    ..( 21 skipped).    ..  **************
189    2019-04-17 09:24    33  **************
190    2019-04-17 09:25    34  ***************
...    ..( 14 skipped).    ..  ***************
205    2019-04-17 09:40    34  ***************
206    2019-04-17 09:41    33  **************
...    ..( 92 skipped).    ..  **************
299    2019-04-17 11:14    33  **************
300    2019-04-17 11:15    32  *************
...    ..(  8 skipped).    ..  *************
309    2019-04-17 11:24    32  *************
310    2019-04-17 11:25    33  **************
...    ..( 18 skipped).    ..  **************
329    2019-04-17 11:44    33  **************
330    2019-04-17 11:45    32  *************
...    ..(  2 skipped).    ..  *************
333    2019-04-17 11:48    32  *************
334    2019-04-17 11:49     ?  -
335    2019-04-17 11:50    25  ******
336    2019-04-17 11:51    25  ******
337    2019-04-17 11:52    25  ******
338    2019-04-17 11:53    26  *******
339    2019-04-17 11:54    26  *******
340    2019-04-17 11:55    27  ********
...    ..(  2 skipped).    ..  ********
343    2019-04-17 11:58    27  ********
344    2019-04-17 11:59    28  *********
...    ..(  5 skipped).    ..  *********
350    2019-04-17 12:05    28  *********
351    2019-04-17 12:06    37  ******************
352    2019-04-17 12:07    37  ******************
353    2019-04-17 12:08    37  ******************
354    2019-04-17 12:09    36  *****************
355    2019-04-17 12:10    36  *****************
356    2019-04-17 12:11    36  *****************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP/SMART Log 0x04) not supported

Pending Defects log (GP Log 0x0c) not supported

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2            3  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x8000  4          972  Vendor specific

nightowl · Apr 17, 2019

root@freenas:~ # smartctl -x /dev/ada2

Code:

smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4N0SN0VL1
LU WWN Device Id: 5 0014ee 2604dfade
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 17 12:13:26 2019 -03
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Unavailable
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever
                    been run.
Total time to complete Offline
data collection:         (40980) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      ( 411) minutes.
Conveyance self-test routine
recommended polling time:      (   5) minutes.
SCT capabilities:            (0x703d)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    1
  3 Spin_Up_Time            POS--K   177   175   021    -    6116
  4 Start_Stop_Count        -O--CK   100   100   000    -    138
  5 Reallocated_Sector_Ct   PO--CK   200   200   140    -    0
  7 Seek_Error_Rate         -OSR-K   200   200   000    -    0
  9 Power_On_Hours          -O--CK   071   071   000    -    21554
10 Spin_Retry_Count        -O--CK   100   100   000    -    0
11 Calibration_Retry_Count -O--CK   100   100   000    -    0
12 Power_Cycle_Count       -O--CK   100   100   000    -    138
192 Power-Off_Retract_Count -O--CK   200   200   000    -    66
193 Load_Cycle_Count        -O--CK   200   200   000    -    383
194 Temperature_Celsius     -O---K   121   101   000    -    29
196 Reallocated_Event_Count -O--CK   200   200   000    -    0
197 Current_Pending_Sector  -O--CK   200   200   000    -    0
198 Offline_Uncorrectable   ----CK   100   253   000    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   200   000    -    0
200 Multi_Zone_Error_Rate   ---R--   100   253   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      6  Ext. Comprehensive SMART error log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa0-0xa7  GPL,SL  VS      16  Device vendor specific log
0xa8-0xb7  GPL,SL  VS       1  Device vendor specific log
0xbd       GPL,SL  VS       1  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL     VS      93  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (6 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       258 (0x0102)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    29 Celsius
Power Cycle Min/Max Temperature:     25/29 Celsius
Lifetime    Min/Max Temperature:      2/49 Celsius
Under/Over Temperature Limit Count:   0/0
Vendor specific:
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -41/85 Celsius
Temperature History Size (Index):    478 (367)

Index    Estimated Time   Temperature Celsius
368    2019-04-17 04:16    36  *****************
...    ..( 12 skipped).    ..  *****************
381    2019-04-17 04:29    36  *****************
382    2019-04-17 04:30    37  ******************
...    ..( 19 skipped).    ..  ******************
402    2019-04-17 04:50    37  ******************
403    2019-04-17 04:51    36  *****************
...    ..( 74 skipped).    ..  *****************
   0    2019-04-17 06:06    36  *****************
   1    2019-04-17 06:07    35  ****************
...    ..( 15 skipped).    ..  ****************
  17    2019-04-17 06:23    35  ****************
  18    2019-04-17 06:24     ?  -
  19    2019-04-17 06:25    35  ****************
...    ..(  3 skipped).    ..  ****************
  23    2019-04-17 06:29    35  ****************
  24    2019-04-17 06:30    36  *****************
...    ..(  8 skipped).    ..  *****************
  33    2019-04-17 06:39    36  *****************
  34    2019-04-17 06:40    35  ****************
...    ..( 23 skipped).    ..  ****************
  58    2019-04-17 07:04    35  ****************
  59    2019-04-17 07:05    34  ***************
...    ..(130 skipped).    ..  ***************
190    2019-04-17 09:16    34  ***************
191    2019-04-17 09:17    33  **************
...    ..(  6 skipped).    ..  **************
198    2019-04-17 09:24    33  **************
199    2019-04-17 09:25    34  ***************
...    ..( 18 skipped).    ..  ***************
218    2019-04-17 09:44    34  ***************
219    2019-04-17 09:45    33  **************
...    ..(122 skipped).    ..  **************
342    2019-04-17 11:48    33  **************
343    2019-04-17 11:49     ?  -
344    2019-04-17 11:50    25  ******
345    2019-04-17 11:51    25  ******
346    2019-04-17 11:52    26  *******
347    2019-04-17 11:53    26  *******
348    2019-04-17 11:54    27  ********
...    ..(  2 skipped).    ..  ********
351    2019-04-17 11:57    27  ********
352    2019-04-17 11:58    28  *********
...    ..(  6 skipped).    ..  *********
359    2019-04-17 12:05    28  *********
360    2019-04-17 12:06    29  **********
361    2019-04-17 12:07    29  **********
362    2019-04-17 12:08    37  ******************
...    ..(  2 skipped).    ..  ******************
365    2019-04-17 12:11    37  ******************
366    2019-04-17 12:12    36  *****************
367    2019-04-17 12:13    36  *****************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP/SMART Log 0x04) not supported

Pending Defects log (GP Log 0x0c) not supported

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2            2  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x8000  4         1088  Vendor specific

nightowl · Apr 17, 2019

root@freenas:~ # smartctl -x /dev/ada3

Code:

smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD30EFRX-68EUZN0
Serial Number:    WD-WCC4NLPERC39
LU WWN Device Id: 5 0014ee 2b58c09b4
Firmware Version: 82.00A82
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 17 12:15:17 2019 -03
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Unavailable
APM feature is:   Unavailable
Rd look-ahead is: Enabled
Write cache is:   Enabled
DSN feature is:   Unavailable
ATA Security is:  Disabled, frozen [SEC2]
Wt Cache Reorder: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever
                    been run.
Total time to complete Offline
data collection:         (40980) seconds.
Offline data collection
capabilities:              (0x7b) SMART execute Offline immediate.
                    Auto Offline data collection on/off support.
                    Suspend Offline collection upon new
                    command.
                    Offline surface scan supported.
                    Self-test supported.
                    Conveyance Self-test supported.
                    Selective Self-test supported.
SMART capabilities:            (0x0003)    Saves SMART data before entering
                    power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      ( 411) minutes.
Conveyance self-test routine
recommended polling time:      (   5) minutes.
SCT capabilities:            (0x703d)    SCT Status supported.
                    SCT Error Recovery Control supported.
                    SCT Feature Control supported.
                    SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    0
  3 Spin_Up_Time            POS--K   179   178   021    -    6033
  4 Start_Stop_Count        -O--CK   100   100   000    -    137
  5 Reallocated_Sector_Ct   PO--CK   200   200   140    -    0
  7 Seek_Error_Rate         -OSR-K   200   200   000    -    0
  9 Power_On_Hours          -O--CK   071   071   000    -    21554
10 Spin_Retry_Count        -O--CK   100   100   000    -    0
11 Calibration_Retry_Count -O--CK   100   100   000    -    0
12 Power_Cycle_Count       -O--CK   100   100   000    -    137
192 Power-Off_Retract_Count -O--CK   200   200   000    -    67
193 Load_Cycle_Count        -O--CK   200   200   000    -    384
194 Temperature_Celsius     -O---K   122   103   000    -    28
196 Reallocated_Event_Count -O--CK   200   200   000    -    0
197 Current_Pending_Sector  -O--CK   200   200   000    -    0
198 Offline_Uncorrectable   ----CK   100   253   000    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   200   000    -    0
200 Multi_Zone_Error_Rate   ---R--   100   253   000    -    0
                            ||||||_ K auto-keep
                            |||||__ C event count
                            ||||___ R error rate
                            |||____ S speed/performance
                            ||_____ O updated online
                            |______ P prefailure warning

General Purpose Log Directory Version 1
SMART           Log Directory Version 1 [multi-sector log support]
Address    Access  R/W   Size  Description
0x00       GPL,SL  R/O      1  Log Directory
0x01           SL  R/O      1  Summary SMART error log
0x02           SL  R/O      5  Comprehensive SMART error log
0x03       GPL     R/O      6  Ext. Comprehensive SMART error log
0x06           SL  R/O      1  SMART self-test log
0x07       GPL     R/O      1  Extended self-test log
0x09           SL  R/W      1  Selective self-test log
0x10       GPL     R/O      1  NCQ Command Error log
0x11       GPL     R/O      1  SATA Phy Event Counters log
0x21       GPL     R/O      1  Write stream error log
0x22       GPL     R/O      1  Read stream error log
0x80-0x9f  GPL,SL  R/W     16  Host vendor specific log
0xa0-0xa7  GPL,SL  VS      16  Device vendor specific log
0xa8-0xb7  GPL,SL  VS       1  Device vendor specific log
0xbd       GPL,SL  VS       1  Device vendor specific log
0xc0       GPL,SL  VS       1  Device vendor specific log
0xc1       GPL     VS      93  Device vendor specific log
0xe0       GPL,SL  R/W      1  SCT Command/Status
0xe1       GPL,SL  R/W      1  SCT Data Transfer

SMART Extended Comprehensive Error Log Version: 1 (6 sectors)
No Errors Logged

SMART Extended Self-test Log Version: 1 (1 sectors)
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

SMART Selective self-test log data structure revision number 1
SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

SCT Status Version:                  3
SCT Version (vendor specific):       258 (0x0102)
SCT Support Level:                   1
Device State:                        Active (0)
Current Temperature:                    28 Celsius
Power Cycle Min/Max Temperature:     24/28 Celsius
Lifetime    Min/Max Temperature:      2/47 Celsius
Under/Over Temperature Limit Count:   0/0
Vendor specific:
01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

SCT Temperature History Version:     2
Temperature Sampling Period:         1 minute
Temperature Logging Interval:        1 minute
Min/Max recommended Temperature:      0/60 Celsius
Min/Max Temperature Limit:           -41/85 Celsius
Temperature History Size (Index):    478 (370)

Index    Estimated Time   Temperature Celsius
371    2019-04-17 04:18    35  ****************
...    ..( 10 skipped).    ..  ****************
382    2019-04-17 04:29    35  ****************
383    2019-04-17 04:30    36  *****************
...    ..( 12 skipped).    ..  *****************
396    2019-04-17 04:43    36  *****************
397    2019-04-17 04:44    35  ****************
...    ..( 80 skipped).    ..  ****************
   0    2019-04-17 06:05    35  ****************
   1    2019-04-17 06:06    34  ***************
...    ..( 16 skipped).    ..  ***************
  18    2019-04-17 06:23    34  ***************
  19    2019-04-17 06:24     ?  -
  20    2019-04-17 06:25    34  ***************
...    ..(  5 skipped).    ..  ***************
  26    2019-04-17 06:31    34  ***************
  27    2019-04-17 06:32    35  ****************
...    ..(  2 skipped).    ..  ****************
  30    2019-04-17 06:35    35  ****************
  31    2019-04-17 06:36    34  ***************
...    ..( 24 skipped).    ..  ***************
  56    2019-04-17 07:01    34  ***************
  57    2019-04-17 07:02    33  **************
...    ..(121 skipped).    ..  **************
179    2019-04-17 09:04    33  **************
180    2019-04-17 09:05    32  *************
...    ..( 18 skipped).    ..  *************
199    2019-04-17 09:24    32  *************
200    2019-04-17 09:25    33  **************
...    ..( 14 skipped).    ..  **************
215    2019-04-17 09:40    33  **************
216    2019-04-17 09:41    32  *************
...    ..(126 skipped).    ..  *************
343    2019-04-17 11:48    32  *************
344    2019-04-17 11:49     ?  -
345    2019-04-17 11:50    24  *****
346    2019-04-17 11:51    24  *****
347    2019-04-17 11:52    24  *****
348    2019-04-17 11:53    25  ******
349    2019-04-17 11:54    26  *******
350    2019-04-17 11:55    26  *******
351    2019-04-17 11:56    27  ********
...    ..(  4 skipped).    ..  ********
356    2019-04-17 12:01    27  ********
357    2019-04-17 12:02    28  *********
...    ..(  6 skipped).    ..  *********
364    2019-04-17 12:09    28  *********
365    2019-04-17 12:10    36  *****************
366    2019-04-17 12:11    35  ****************
...    ..(  3 skipped).    ..  ****************
370    2019-04-17 12:15    35  ****************

SCT Error Recovery Control:
           Read:     70 (7.0 seconds)
          Write:     70 (7.0 seconds)

Device Statistics (GP/SMART Log 0x04) not supported

Pending Defects log (GP Log 0x0c) not supported

SATA Phy Event Counters (GP Log 0x11)
ID      Size     Value  Description
0x0001  2            0  Command failed due to ICRC error
0x0002  2            0  R_ERR response for data FIS
0x0003  2            0  R_ERR response for device-to-host data FIS
0x0004  2            0  R_ERR response for host-to-device data FIS
0x0005  2            0  R_ERR response for non-data FIS
0x0006  2            0  R_ERR response for device-to-host non-data FIS
0x0007  2            0  R_ERR response for host-to-device non-data FIS
0x0008  2            0  Device-to-host non-data FIS retries
0x0009  2            2  Transition from drive PhyRdy to drive PhyNRdy
0x000a  2            3  Device-to-host register FISes sent due to a COMRESET
0x000b  2            0  CRC errors within host-to-device FIS
0x000f  2            0  R_ERR response for host-to-device data FIS, CRC
0x0012  2            0  R_ERR response for host-to-device non-data FIS, CRC
0x8000  4         1199  Vendor specific

nightowl · Apr 17, 2019

Code:

root@freenas:~ # glabel status
                                      Name  Status  Components
gptid/8f3569c9-7489-11e5-8705-d0509913e8a9     N/A  da0p1
gptid/6a45ab6d-1140-11e6-980e-d0509913e8a9     N/A  ada0p2
gptid/695ab06c-1140-11e6-980e-d0509913e8a9     N/A  ada1p2
gptid/6c49e955-1140-11e6-980e-d0509913e8a9     N/A  ada2p2
gptid/6b64892f-1140-11e6-980e-d0509913e8a9     N/A  ada3p2

SweetAndLow · Apr 17, 2019

Run a long smart test on all your drives. You have some with read failures so those might be dead.

Important Announcement for the TrueNAS Community.

Recover

Dabbler

Hall of Famer

Dabbler

Dabbler

Hall of Famer

Dabbler

Sweet'NASty

Patron

Hall of Famer

Hall of Famer

Dabbler

Hall of Famer

Hall of Famer

Dabbler

Dabbler

Dabbler

Dabbler

Dabbler

Dabbler

Sweet'NASty

Important Announcement for the TrueNAS Community.

Related topics on forums.truenas.com for thread: "Recover"

Similar threads