Hey all.
So for the past three day's I've woke up to a few errors in my nightly FreeNAS security output email. I'm not sure if this is just a SATA Cable issue or a drive issue.
The system is an
HP XW8600
Dual Xeon quad core E5440
24 GB DDR2 5300F ECC
4 x WD RED 2TB Drives RAIDz
FreeNAS 9.1.1 Release-x64
This is what I noticed and am unsure on how to proceed with it.
The Security Run Output email shows the following over and over again, around 75 times each
Running zpool status -v shows the following:
The Alert System icon in the Freenas GUI this morning was green, then changed colour a little later on saying: One or more devices has experienced an unrecoverable error.
I shut the system down and checked the SATA cable for /dev/ada1 and rebooted the system. The GUI Alert System Icon is now reporting everything is OK again.
Running smartctl -a /dev/ada1 shows the following SMART data:
Usually when I transfer a large amount of data from the FreeNAS box to another system I can reach and sustain around 90 Mb/s in transfer speed. I've noticed that this has stopped and I'm only getting between 10 and 25 Mb/s now.
Is this a drive failure, or a SATA cable issue? If it is just a SATA Cable issue, should I be taking the drive out, formatting it and place it back in the zpool and re-sliver it? If its a drive issue, its under warranty still so I'll go pickup another and to replace it and send this one in for a warranty replacement, and toss that replacement into another pool that I'm about to build.
So for the past three day's I've woke up to a few errors in my nightly FreeNAS security output email. I'm not sure if this is just a SATA Cable issue or a drive issue.
The system is an
HP XW8600
Dual Xeon quad core E5440
24 GB DDR2 5300F ECC
4 x WD RED 2TB Drives RAIDz
FreeNAS 9.1.1 Release-x64
This is what I noticed and am unsure on how to proceed with it.
The Security Run Output email shows the following over and over again, around 75 times each
Code:
freenas.local kernel log messages: +++ /tmp/security.2KfJlu2f 2014-03-29 03:01:00.000000000 -0600 +(ada1:ahcich1:0:0:0): READ_FPDMA_QUEUED. ACB: 60 50 a8 ed 7b 40 3a 00 00 00 00 00 +(ada1:ahcich1:0:0:0): CAM status: Uncorrectable parity/CRC error +(ada1:ahcich1:0:0:0): Retrying command
Running zpool status -v shows the following:
Code:
~# zpool status -v
pool: MEDIA-NAS
state: ONLINE
status: The pool is formatted using a legacy on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
pool will no longer be accessible on software that does not support feature
flags.
scan: scrub repaired 4K in 5h19m with 0 errors on Sat Mar 29 06:19:18 2014
config:
NAME STATE READ WRITE CKSUM
MEDIA-NAS ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
gptid/d7cd16c8-bdc0-11e2-9508-002264220f03 ONLINE 0 0 0
gptid/d83a132e-bdc0-11e2-9508-002264220f03 ONLINE 0 0 0
gptid/d8a5f48e-bdc0-11e2-9508-002264220f03 ONLINE 0 0 0
gptid/d91cd99c-bdc0-11e2-9508-002264220f03 ONLINE 0 0 0
errors: No known data errors
The Alert System icon in the Freenas GUI this morning was green, then changed colour a little later on saying: One or more devices has experienced an unrecoverable error.
I shut the system down and checked the SATA cable for /dev/ada1 and rebooted the system. The GUI Alert System Icon is now reporting everything is OK again.
Running smartctl -a /dev/ada1 shows the following SMART data:
Code:
~# smartctl -a /dev/ada1
smartctl 6.1 2013-03-16 r3800 [FreeBSD 9.1-STABLE amd64] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital Red (AF)
Device Model: WDC WD20EFRX-68AX9N0
Serial Number: WD-WMC300701078
LU WWN Device Id: 5 0014ee 058d0491e
Firmware Version: 80.00A80
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Sector Sizes: 512 bytes logical, 4096 bytes physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ACS-2 (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Sat Mar 29 15:04:11 2014 MDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (27120) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 274) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x70bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 177 175 021 Pre-fail Always - 4108
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 208
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0
9 Power_On_Hours 0x0032 088 088 000 Old_age Always - 9437
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 187
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 145
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 62
194 Temperature_Celsius 0x0022 115 105 000 Old_age Always - 32
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 14
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 9378 -
# 2 Short offline Completed without error 00% 9234 -
# 3 Extended offline Completed without error 00% 9141 -
# 4 Short offline Interrupted (host reset) 10% 9070 -
# 5 Short offline Completed without error 00% 8900 -
# 6 Extended offline Completed without error 00% 8783 -
# 7 Short offline Completed without error 00% 8756 -
# 8 Short offline Completed without error 00% 8708 -
# 9 Short offline Completed without error 00% 8564 -
#10 Extended offline Completed without error 00% 8471 -
#11 Short offline Completed without error 00% 8413 -
#12 Short offline Completed without error 00% 8228 -
#13 Extended offline Completed without error 00% 8119 -
#14 Short offline Completed without error 00% 8092 -
#15 Short offline Completed without error 00% 7972 -
#16 Short offline Completed without error 00% 7829 -
#17 Extended offline Completed without error 00% 7736 -
#18 Short offline Completed without error 00% 7673 -
#19 Short offline Completed without error 00% 7493 -
#20 Extended offline Completed without error 00% 7376 -
#21 Short offline Completed without error 00% 7351 -
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Usually when I transfer a large amount of data from the FreeNAS box to another system I can reach and sustain around 90 Mb/s in transfer speed. I've noticed that this has stopped and I'm only getting between 10 and 25 Mb/s now.
Is this a drive failure, or a SATA cable issue? If it is just a SATA Cable issue, should I be taking the drive out, formatting it and place it back in the zpool and re-sliver it? If its a drive issue, its under warranty still so I'll go pickup another and to replace it and send this one in for a warranty replacement, and toss that replacement into another pool that I'm about to build.
