It depends on if you upgraded the pool to the latest version each time. When I upgrade my FreeNAS versions, I do not upgrade my pools anymore so I can move backwards. If you wanted to use a 9.3 version 6 months old or so, that should be fine.Can it help to build a earlier version of freenas.
[root@freenas] ~# smartctl -a /dev/ada4 smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p28 amd64] (local build) Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Western Digital Caviar Green (AF, SATA 6Gb/s) Device Model: WDC WD20EARX-00PASB0 Serial Number: WD-WCAZAE112434 LU WWN Device Id: 5 0014ee 207050db1 Firmware Version: 51.0AB51 User Capacity: 2,000,398,934,016 bytes [2.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Device is: In smartctl database [for details use: -P show] ATA Version is: ATA8-ACS (minor revision not indicated) SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Wed Jan 20 10:07:05 2016 PST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (39300) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 379) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3035) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 1 3 Spin_Up_Time 0x0027 175 168 021 Pre-fail Always - 6233 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 348 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 076 076 000 Old_age Always - 17864 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 346 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 304 193 Load_Cycle_Count 0x0032 184 184 000 Old_age Always - 48621 194 Temperature_Celsius 0x0022 112 093 000 Old_age Always - 38 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 1 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 17820 - # 2 Short offline Completed without error 00% 17777 - # 3 Short offline Completed without error 00% 17775 - # 4 Short offline Completed without error 00% 17774 - # 5 Short offline Completed without error 00% 17773 - # 6 Short offline Completed without error 00% 17772 - # 7 Short offline Completed without error 00% 17771 - # 8 Short offline Completed without error 00% 17770 - # 9 Short offline Completed without error 00% 17769 - #10 Short offline Completed without error 00% 17768 - #11 Short offline Completed without error 00% 17767 - #12 Short offline Completed without error 00% 17766 - #13 Short offline Completed without error 00% 17765 - #14 Short offline Completed without error 00% 17764 - #15 Short offline Completed without error 00% 17763 - #16 Short offline Completed without error 00% 17762 - #17 Short offline Completed without error 00% 17761 - #18 Short offline Completed without error 00% 17760 - #19 Short offline Completed without error 00% 17759 - #20 Short offline Completed without error 00% 17758 - #21 Short offline Completed without error 00% 17757 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.
Serial Number: WD-WCAWZ1695717
1 Extended offline Completed: read failure 20% 29947 3484181056
[root@freenas] ~# smartctl -a /dev/ada1 smartctl 6.4 2015-06-04 r4109 [FreeBSD 10.2-STABLE amd64] (local build) Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Western Digital Caviar Green (AF) Device Model: WDC WD20EARS-00J99B0 Serial Number: WD-WCAWZ1695717 LU WWN Device Id: 5 0014ee 6ac887eae Firmware Version: 80.00A80 User Capacity: 2,000,398,934,016 bytes [2.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Device is: In smartctl database [for details use: -P show] ATA Version is: ATA8-ACS (minor revision not indicated) SATA Version is: SATA 2.6, 3.0 Gb/s Local Time is: Wed Jan 20 20:43:25 2016 PST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 114) The previous self-test completed having the read element of the test failed. Total time to complete Offline data collection: (34080) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 329) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3035) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 160 134 021 Pre-fail Always - 8966 4 Start_Stop_Count 0x0032 098 098 000 Old_age Always - 2100 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 059 059 000 Old_age Always - 30001 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 906 192 Power-Off_Retract_Count 0x0032 199 199 000 Old_age Always - 823 193 Load_Cycle_Count 0x0032 037 037 000 Old_age Always - 489981 194 Temperature_Celsius 0x0022 113 091 000 Old_age Always - 39 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 1 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed: read failure 20% 29998 3484181056 # 2 Extended offline Completed: read failure 20% 29984 3484181056 # 3 Extended offline Completed: read failure 20% 29947 3484181056 # 4 Short offline Completed without error 00% 29905 - # 5 Short offline Completed without error 00% 29904 - # 6 Short offline Completed without error 00% 29903 - # 7 Short offline Completed without error 00% 29902 - # 8 Short offline Completed without error 00% 29901 - # 9 Short offline Completed without error 00% 29900 - #10 Short offline Completed without error 00% 29899 - #11 Short offline Completed without error 00% 29898 - #12 Short offline Completed without error 00% 29897 - #13 Short offline Completed without error 00% 29896 - #14 Short offline Completed without error 00% 29895 - #15 Short offline Completed without error 00% 29894 - #16 Short offline Completed without error 00% 29893 - #17 Short offline Completed without error 00% 29892 - #18 Short offline Completed without error 00% 29891 - #19 Short offline Completed without error 00% 29890 - #20 Short offline Completed without error 00% 29889 - #21 Short offline Completed without error 00% 29888 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.
The fact that it's consistently failing SMART self-tests is, IMO, a surefire indication.Out of interest what told you the drive was bad?
This is the result for all disks with:
Code:zdb -l /dev/ada4
I am not getting a good feeling about this
This is expected. ZFS (normally) lives in the p2 partitions of your drives. And your gpart show does show the p2 partitions. Try "zdb -l /dev/adaXp2".
[root@freenas] ~# zdb -l /dev/ada3 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~# zdb -l /dev/ada2 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~# zdb -l /dev/ada1 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~# zdb -l /dev/ada0 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~#
That's not what he asked. He asked for "zdb -l /dev/adaXp2".[root@freenas] ~# zdb -l /dev/ada4
[root@freenas] ~# zdb -l /dev/ada0p2 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~# zdb -l /dev/ada1p2 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~# zdb -l /dev/ada2p2 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~# zdb -l /dev/ada3p2 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~# zdb -l /dev/ada4p2 -------------------------------------------- LABEL 0 -------------------------------------------- failed to unpack label 0 -------------------------------------------- LABEL 1 -------------------------------------------- failed to unpack label 1 -------------------------------------------- LABEL 2 -------------------------------------------- failed to unpack label 2 -------------------------------------------- LABEL 3 -------------------------------------------- failed to unpack label 3 [root@freenas] ~#
dd if=/dev/da0p2 | strings | grep pool_guid
root@nas ~ # dd if=/dev/da0p2 | strings | grep pool_guid pool_guid pool_guid
root@nas ~ # dd if=/dev/da0p2 bs=1k skip=128 count=1 | hd 1+0 records in 1+0 records out 1024 bytes transferred in 0.003580 secs (286045 bytes/sec) 00000000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................|
root@nas ~ # dd if=/dev/da2p2 bs=1k | hd | grep "0c b1 ba 00" 00020000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00021000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00022000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00023000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00024000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00025000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00026000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00027000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00028000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00029000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 0002a000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 0002b000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 0002c000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 0002d000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 0002e000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 0002f000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................| 00030000 0c b1 ba 00 00 00 00 00 88 13 00 00 00 00 00 00 |................|