Pool is offline. i/o failure. don't use sata to pci 1x cards.. Drives are fine.

CookieMonster1 · Oct 14, 2022

4x16tb drives z2.
I woke up a few days ago the pool is offline. These are data center drives really new. I have really important data such as school stuff on here and vms.
WARNING
New ZFS version or feature flags are available for pool(s) data1. Upgrading pools is a one-time process that can prevent rolling the system back to an earlier TrueNAS version. It is recommended to read the TrueNAS release notes and confirm you need the new ZFS feature flags before upgrading a pool.
2022-10-05 13:06:11 (America/Los_Angeles)

I did upgrade the pool, I thought but maybe not. Have another pool and its fine. I did only upgrade both pools at the same time before. So not sure why this poped up. Always do both. This was before the pool went offline.
When I try to create a pool I can wipe the drives. I can't do that.. I have other computers laying around so I could spin up another truenas on a older version then upgrade it the pool with the same hardware. Would that fix my problems?

This christmas I am going to try and get an offsite backup. Just don't have the money right now.

Going to run a normal smart test for long. I do read the RTFM. Hardware is not supported but I have no choice. I'm broke. The drives alone cost $1,200..
My hard drives are connect through sata to pci. 2x cards. 4x sata cables. Total of 12 drives running. Have been running for 1 year now fine.

danb35 · Oct 14, 2022

CookieMonster1 said:
When I try to create a pool I can wipe the drives.

You certainly wouldn't want to do that--either of those, really; you wouldn't want to create a pool or wipe the drives. I'm afraid I have no idea what you're quoting with the "remaining 0, lifetime 6146" stuff, or how you have "4x16tb drives z2" and "Total of 12 drives running."

I understand from your post that a pool is offline. What happens if you try to import the pool through the GUI? What do you see if you run zpool import at the CLI?

CookieMonster1 · Oct 14, 2022

Through the gui. I can't select any drive. I tried yes and no for encryption.

Mine are encrypted.
Nothing worked.
What are the commands to import a pool. It will be about a day until I have enough time to play around with it again. Will message back when I can.

CookieMonster1 · Oct 14, 2022

I would try and export disconnect the pool but I don't have any backups for NFS/SMB's. Would be faster if I spin up a new server and remove the drives. Have lots of shares.
I know it probably won't fix it because this is a software issue.

Davvo · Oct 14, 2022

If the drives are failing smart tests it's not a software issue.
HD model? Please comply with the forum rules.

CookieMonster1 said:
My hard drives are connect through sata to pci. 2x cards. 4x sata cables. Total of 12 drives running. Have been running for 1 year now fine.

Multiply your problems with SATA Port Multipliers and cheap SATA controllers

In the last year or two, we've had a resurgence of users asking about SATA Port Multipliers and cheap SATA controllers. Please, do NOT use port multipliers, and use cheap SATA controllers only after extensive research. SATA controllers and SATA...

www.truenas.com

joeschmuck · Oct 14, 2022

So my two cents are:
1) You are not clear on describing the errors other than your pool is offline.
2) All the drive data you mentioned really is out of context. Slow down and if you desire to post the entire SMART output of all your drives, make sure you use CODE brackets, but I highly doubt your drives are the issue.
3) Have you rebooted?
4) Do you have a backup of your TrueNAS configuration file?
5) Do not upgrade your ZFS Pool !!! Not until after you recover your pool.
6) What is your boot drive? A Flash Drive, SSD, Hard Drive?
7) If you can, in the GUI, can you select the previous Boot Environment?
8) You could also try to recreate a new boot drive if you must but I'd save that for last. I dislike working with encrypted drives so you should ensure you get some good advice on this aspect.
9) Do you think you could provide some information about your system and it's setup/configuration per the forum rules? It will help us from just guessing. Computer (CPU/RAM/Boot Drive), Hard Drives, HBA if you have one.

CookieMonster1 · Oct 16, 2022

Specs:
Dell 7010. Custom case with 12 ssd/hdd drives bays.
Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
16GB ram
2x4 TB - ST4000DM004-2CV104
4x8 TB ST8000DM004-2CX188
4x16 TB - ST16000NM000J-2TW103 Having problem with this pool. Z2.
2x120 GB dual boot ssd's - 1 is a backup.
500 Watt power supply.

As soon as I get up and running I am going to copy all my data to 4x8 TB just in case. Have 4TB of data that I really need. Years of stuff.

1) I don't know why my pool is offline. I updated my truenas to the lastest update. The pools were fine for over 2 weeks at least. Then it went offline. I am able to wipe the drives but not able to do that.

2) What is the symbol for a code brackets. Sorry.

3) I have rebooted the computer.

4) Have not upgraded my pool yet. Has been offline.

7) I am not sure. Where do I find that at?

8) I have another 7010 same as before without a custom case. I can load another truenas version of choice and try and import the pool. I don't really have the time right now but I can in the upcoming days.
I just got some time today to write this. 1 smart test has failed for long. Other 3 drives seem to be fine. Z2. Will get a replacement after I copy all the data off first to 4x8 TB ST8000DM004-2CX188.

I haven't learned any truenas command yet. So if want me to run something let me know. Thanks.

CookieMonster1 · Oct 16, 2022

I want to say thank you for everyone that has tried to help me so far. I do really mean it!

CookieMonster1 · Oct 16, 2022

[root@truenas[~]# smartctl -a /dev/ada6
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p1 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, [URL='http://www.smartmontools.org']www.smartmontools.org[/URL]

=== START OF INFORMATION SECTION ===
Device Model:     ST16000NM000J-2TW103
Serial Number:   
LU WWN Device Id: 5 000c50 0dc35bde6
Firmware Version: SN02
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sun Oct 16 19:16:09 2022 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  559) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1355) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x70bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   074   064   044    Pre-fail  Always       -       24256947
  3 Spin_Up_Time            0x0003   093   091   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       458
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   077   060   045    Pre-fail  Always       -       52607030
  9 Power_On_Hours          0x0032   093   093   000    Old_age   Always       -       6216
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       458
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   060   049   000    Old_age   Always       -       40 (Min/Max 35/51)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       439
193 Load_Cycle_Count        0x0032   067   067   000    Old_age   Always       -       66558
194 Temperature_Celsius     0x0022   040   051   000    Old_age   Always       -       40 (0 22 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       492 (18 192 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       6720236852
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       204901812473

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%      6214         -
# 2  Extended offline    Interrupted (host reset)      00%      6146         -
# 3  Short offline       Completed without error       00%      6142         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

root@truenas[~]# smartctl -a /dev/ada4
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p1 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, [URL='http://www.smartmontools.org']www.smartmontools.org[/URL]

=== START OF INFORMATION SECTION ===
Device Model:     ST16000NM000J-2TW103
Serial Number:   
LU WWN Device Id: 5 000c50 0dc762c06
Firmware Version: SN02
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sun Oct 16 19:16:21 2022 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  567) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1422) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x70bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   082   064   044    Pre-fail  Always       -       176721801
  3 Spin_Up_Time            0x0003   094   091   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   097   097   020    Old_age   Always       -       3653
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   077   060   045    Pre-fail  Always       -       55584096
  9 Power_On_Hours          0x0032   093   093   000    Old_age   Always       -       6216
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   097   097   020    Old_age   Always       -       3654
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   099   000    Old_age   Always       -       1
190 Airflow_Temperature_Cel 0x0022   061   052   000    Old_age   Always       -       39 (Min/Max 34/48)
192 Power-Off_Retract_Count 0x0032   099   099   000    Old_age   Always       -       3634
193 Load_Cycle_Count        0x0032   066   066   000    Old_age   Always       -       69709
194 Temperature_Celsius     0x0022   039   048   000    Old_age   Always       -       39 (0 22 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       490 (156 195 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       6720092076
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       165433952090

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%      6215         -
# 2  Extended offline    Interrupted (host reset)      00%      6146         -
# 3  Short offline       Completed without error       00%      6141         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

root@truenas[~]# smartctl -a /dev/ada1
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p1 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, [URL='http://www.smartmontools.org']www.smartmontools.org[/URL]

=== START OF INFORMATION SECTION ===
Device Model:     ST16000NM000J-2TW103
Serial Number:   
LU WWN Device Id: 5 000c50 0dc35b569
Firmware Version: SN02
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sun Oct 16 19:16:23 2022 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  559) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1375) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x70bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   080   064   044    Pre-fail  Always       -       96459821
  3 Spin_Up_Time            0x0003   096   091   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   086   086   020    Old_age   Always       -       14987
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   073   060   045    Pre-fail  Always       -       20891969
  9 Power_On_Hours          0x0032   093   093   000    Old_age   Always       -       6213
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   086   086   020    Old_age   Always       -       14984
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   059   049   000    Old_age   Always       -       41 (Min/Max 40/48)
192 Power-Off_Retract_Count 0x0032   093   093   000    Old_age   Always       -       14964
193 Load_Cycle_Count        0x0032   060   060   000    Old_age   Always       -       81702
194 Temperature_Celsius     0x0022   041   049   000    Old_age   Always       -       41 (0 22 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       315 (20 166 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       25972827276
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       115337743932

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Interrupted (host reset)      00%      6191         -
# 2  Extended offline    Interrupted (host reset)      00%      6139         -
# 3  Short offline       Interrupted (host reset)      00%      6139         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing



root@truenas[~]# smartctl -a /dev/ada8
smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RELEASE-p1 amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, [URL='http://www.smartmontools.org']www.smartmontools.org[/URL]

=== START OF INFORMATION SECTION ===
Device Model:     ST16000NM000J-2TW103
Serial Number:    ZR701YSW
LU WWN Device Id: 5 000c50 0dc35be6d
Firmware Version: SN02
User Capacity:    16,000,900,661,248 bytes [16.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-4 (minor revision not indicated)
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sun Oct 16 19:19:47 2022 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (  41) The self-test routine was interrupted
                                        by the host with a hard or soft reset.
Total time to complete Offline
data collection:                (  559) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (1349) minutes.
Conveyance self-test routine
recommended polling time:        (   2) minutes.
SCT capabilities:              (0x70bd) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   082   064   044    Pre-fail  Always       -       142552571
  3 Spin_Up_Time            0x0003   093   091   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       80
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   078   060   045    Pre-fail  Always       -       68662971
  9 Power_On_Hours          0x0032   093   093   000    Old_age   Always       -       6216
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       81
 18 Unknown_Attribute       0x000b   100   100   050    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
188 Command_Timeout         0x0032   100   099   000    Old_age   Always       -       1
190 Airflow_Temperature_Cel 0x0022   059   048   000    Old_age   Always       -       41 (Min/Max 40/50)
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       61
193 Load_Cycle_Count        0x0032   068   068   000    Old_age   Always       -       65873
194 Temperature_Celsius     0x0022   041   052   000    Old_age   Always       -       41 (0 22 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0023   100   100   001    Pre-fail  Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       539 (252 128 0)
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       6720288140
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       213626099720

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Interrupted (host reset)      00%      6195         -
# 2  Extended offline    Completed without error       00%      6162         -
# 3  Short offline       Completed without error       00%      6142         -
# 4  Extended offline    Interrupted (host reset)      00%      5400         -
# 5  Extended offline    Completed without error       00%      5183         -
# 6  Extended offline    Completed without error       00%      4982         -
# 7  Short offline       Completed without error       00%      4652         -
# 8  Short offline       Completed without error       00%      4651         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing]

Davvo · Oct 17, 2022

The right tags are [CODE][/CODE]

danb35 · Oct 17, 2022

I asked on Friday for the output of zpool import, but haven't yet seen it. Your disks are looking OK. Warmer than I'd like to see, and the lack of scheduled SMART self-tests is a concern, but neither of those would keep the pool from being available.

joeschmuck · Oct 17, 2022

I agree, the disks look okay but WOW! some of them have more power cycles that hours, one significantly more. ada1 has had it's power cycled 2.4 times for each hour it's been on. Do you have an electrical problem? Connectors not fully mated? Bad connectors or maybe a bad power supply? It just strikes me as odd. ada4 and ada6 also have notable power cycle values.

Maybe i have it wrong and someone can correct me. And this may have nothing to do with your pool status.

ChrisRJ · Oct 17, 2022

Those power cycle counts are indeed a problem, especially since we are talking about Seagate Exos, which are data center drives. I seem to remember that someone once wrote here that those drives come with a better warranty, but are otherwise the same as other drives. No idea, whether this is correct or not. But I distinctly remember to have read in summer 2020 in a Synology support document, that automatic spin down was explicitly not supported with Exos drives, while at the same time it was for Ironwolf (Pro). I am inclined to consider this relevant.

CookieMonster1 · Oct 17, 2022

You are making me worried about the power cycles then hours on. How do I change that? I left everything the default.

How do I do the zpool import through the terminal?

I don't really use the drives that much. Its just long term storage. Am I ruining the drives?

I will also send the other pool drives in the morning. Just in case. They all do the same thing.

joeschmuck · Oct 18, 2022

Honestly I have no idea why the Power Cycles are doing that. That normally indicates the number of times that power has been applied to the drive. It has nothing to do with sleeping the drive or anything else. This is why I was suggesting for you to examine the power connectors on the drives, make sure they are fully seated. If yo have a backplane they plug into, maybe the backplane is bad. If they are plugged in okay then maybe you have a bad power supply. Or maybe the 10 hard drives and your motherboard are pulling more power than you think. A 500 Watt power supply may not be enough. This is the point where you add up all your power requirements for +5VDC and +12VDC and see if the power supply is capable of handling this current.

If the drives are spinning up and down often then yes, you are likely harming the drives in the long run but that is just my opinion.

cobrakiller58 · Oct 18, 2022

CookieMonster1 said:
Through the gui. I can't select any drive. I tried yes and no for encryption.

Mine are encrypted.
Nothing worked.
What are the commands to import a pool. It will be about a day until I have enough time to play around with it again. Will message back when I can.

Your pool was encrypted before you lost access to it?

danb35 · Oct 18, 2022

joeschmuck said:
Maybe i have it wrong

I was in the middle of typing that I thought you were looking at the load cycle count (which is also quite high) when I took a closer look, and something that strikes me is the inconsistency here. ada6 shows a power cycle count of 458. ada4, 3654. ada1, 14,984. ada8, 81 (with a power-off retract count of only 61 somehow). All with the same power-on hours, +/- a few hours.

CookieMonster1 said:
How do I do the zpool import through the terminal?

Um, type zpool import and press Enter?

CookieMonster1 · Oct 18, 2022

It was unlocked. Yes they are encrypted drives. I can do a RMA for ada1, 14,984 after this is all fixed. Is it worth it? Or should I wait until the drives dies?

CookieMonster1 · Oct 18, 2022

danb35 said:
I was in the middle of typing that I thought you were looking at the load cycle count (which is also quite high) when I took a closer look, and something that strikes me is the inconsistency here. ada6 shows a power cycle count of 458. ada4, 3654. ada1, 14,984. ada8, 81 (with a power-off retract count of only 61 somehow). All with the same power-on hours, +/- a few hours.

Um, type zpool import and press Enter?

NOT SUPPORTED AND WILL RESULT IN UNDEFINED BEHAVIOR AND MAY
RESULT IN SYSTEM FAILURE.

root@truenas[~]# zpool import
   pool: Rabbit2 - That's the pool, I need.
     id: 11485786615206464425
  state: ONLINE
status: One or more devices were being resilvered.
 action: The pool can be imported using its name or numeric identifier.
 config:

CookieMonster1 · Oct 18, 2022

Is it possible that it could be the motherboard? Its from 2013 was never designed for this.

Important Announcement for the TrueNAS Community.

Pool is offline. i/o failure. don't use sata to pci 1x cards.. Drives are fine.

Explorer

Hall of Famer

Explorer

Explorer

MVP

Old Man

Explorer

Explorer

Explorer

MVP

Hall of Famer

Old Man

Wizard

Explorer

Old Man

Guru

Hall of Famer

Explorer

Explorer

Explorer

Similar threads