WD Red (WD60EFRX-68M) life seems to be about 40,000 Hours (What is your experience?)

NASbox · Aug 20, 2020

For what it's worth, the older WD RED WDC (WD60EFRX-68M) seems to be about 40,000 Hours. I have 8 of these puppies, with staggered run times and different manufacturing lots, and I am seeing the second drive to hit the ~40,000 Hour mark beginning to show pending sectors. ( Also had one go within the 3 year warranty that WD RMA's very quickly/easily).

(I bought bigger drives to replace smaller drives for cash flow reasons - extra space was wasted for awhile, but it was less costly than having left over small drives. It allowed me to ease into a storage upgrade and gave me a pool with staggered manufacturing batches and run times to reduce the statistical odds of multiple simultaneous failures)

Due to WD changing to SMR on the 6TB Reds, I have been replacing my failed 6TB Reds with 8TB WD80EFAX which is CMR.

I'm wondering what the experience of the community is?

Evertb1 · Aug 23, 2020

I just replaced a set of 6 2TB WD Reds who were running for around 43,000 to 44,000 hours, say five years. One of those disks was starting to produce some problems, the others were still running without any problems. However, it has always been my plan to keep the disks for max five years and then replace them. I try to plan my needs for storage capacity accordingly to the expected use of five years. By the way: in order to try to reduce the chance on batch related problems, I ordered the replacing disks (Ironwolfs) at three different resellers.

NASbox · Aug 23, 2020

Evertb1 said:
I just replaced a set of 6 2TB WD Reds who were running for around 43,000 to 44,000 hours, say five years. One of those disks was starting to produce some problems, the others were still running without any problems. However, it has always been my plan to keep the disks for max five years and then replace them. I try to plan my needs for storage capacity accordingly to the expected use of five years. By the way: in order to try to reduce the chance on batch related problems, I ordered the replacing disks (Ironwolfs) at three different resellers.

Thanks for the input @Evertb1.

Do you burn in your new drives? I typically run badblocks followed by a long scan. I just installed an 8TB drive (da7) and that took about 5-6 days to burn in..

When I upgraded my pool fom I rebuilt from 4x3TB to an 8 drive pool. I bought 4 new 6TB drives which I burned in before using, and kepth the original 3TB drives and then replaced the others over time when I found a decent sale. So I have 2 cohorts of 4 drives. One of the later cohort failed after about 2 years and was replaced under warranty (da6). da7 failed at the 40,000 Hr mark and was replaced with an 8TB drive, and da4 is just starting to develop a few pending sectors, so it will be the next one replaced. My plan is to replace everything with 8TB (or 10TB if I get a good sale) drives over the next year or so,

You make a good point about manufacturing lots. Were I live, I used a local supplier who was willing to confirm manufacturing dates. I was fortunate to be able to insist that no 2 drives were built in the same month, and they are staggered over time.

Code:

DEV   Model                             Hours  Yrs  Days
da0   WDC_WD60EFRX-68L0BN1              22933   2 225.54
da1   WDC_WD60EFRX-68L0BN1              22884   2 223.50
da2   WDC_WD60EFRX-68MYMN1              41167   4 255.29
da3   WDC_WD60EFRX-68MYMN1              41907   4 286.12
da4   WDC_WD60EFRX-68MYMN1              40256   4 217.33
da5   WDC_WD60EFRX-68L0BN1              22698   2 215.75
da6   WDC_WD60EFRX-68L0BN1               6673   0 278.04
da7   WDC_WD80EFAX-68KNBN0                826   0  34.42

Evertb1 · Aug 24, 2020

NASbox said:
Do you burn in your new drives?

Yes I burn in my new drives. I have a lab computer running ESXi with a FreeNAS VM where I can plug in extra drives. The HBA I use is passed through to that FreeNAS VM. So the whole SMART test and badblocks thing I can do on that VM. By the way, I start with a short and long self-test. Of course after badblocks I run the long self-test again.

NASbox said:
Were I live, I used a local supplier who was willing to confirm manufacturing dates.

I use suppliers that are reasonable big outfits and are buying large numbers. So the chance that the different suppliers have drives of the same batch is pretty small. Looking at the serial numbers and production dates (I checked) I am pretty sure I have disks from five different batches. Two disks from one supplier were from the same batch.

Wolfeman0101 · Sep 1, 2020

Yeah 40K and I know they are going to start being unreliable.

NASbox · Sep 1, 2020

Wolfeman0101 said:
Yeah 40K and I know they are going to start being unreliable.

Have you had a lot of experience with WD Reds, or is this just a comment about hard drives in general?

Wolfeman0101 · Sep 1, 2020

NASbox said:
Have you had a lot of experience with WD Reds, or is this just a comment about hard drives in general?

I've been using Reds exclusively for a long time. Longer than 40K hours :)

NASbox · Sep 1, 2020

Wolfeman0101 said:
I've been using Reds exclusively for a long time. Longer than 40K hours :)

Approx how many drives have you used over the years, and how many have you had to replalace (or did you just choose to swap them out/upgrade to larger model)?

Wolfeman0101 · Sep 1, 2020

I've got a 6 drive FreeNAS running Red 4TBs right now. I've just replaced 4TBs with 4TBs when they went bad but I'm going to start getting lareger drives. da2 actually started throwing read errors last night so I need to replace that. 4 years 10 months ~ 41760 hours. da0 and da1 where warranted and da5 I just had to replace 9 months ago.

Device	Serial Number	SMART Status	Temp	Power-On Time	Start/Stop Count	Spin Retry Count	Realloc'd Sectors	Realloc Events	Current Pending Sectors	Offline Uncorrectable Sectors	UltraDMA CRC Errors	Seek Error Health	Last Test Age (days)	Last Test Type
SMART Status Report Summary
/dev/da0	WD-WCC7K0XNAULA	PASSED	33*C	3y 0m 17d 10h	49	0	0	0	0	0	0	100%	0	Short
/dev/da1	WD-WCC4E6HZTD6X	PASSED	35*C	2y 11m 9d 5h	50	0	0	0	0	0	0	100%	0	Short
/dev/da2	WD-WCC4E7SD0Z2F	PASSED	34*C	4y 10m 22d 12h	74	0	0	0	0	0	0	100%	0	Short
/dev/da3	WD-WCC4E2YZNDX7	PASSED	36*C	4y 10m 22d 11h	72	0	0	0	0	0	0	100%	0	Short
/dev/da4	WD-WCC4E5KJ3Y0L	PASSED	37*C	4y 10m 22d 11h	71	0	0	0	0	0	0	100%	0	Short
/dev/da5	WD-WCC7K1DF3RRY	PASSED	34*C	0y 9m 16d 3h	9	0	0	0	0	0	0	100%	0	Short

Apollo · Sep 1, 2020

Wolfeman0101 said:
I've got a 6 drive FreeNAS running Red 4TBs right now. I've just replaced 4TBs with 4TBs when they went bad but I'm going to start getting lareger drives. da2 actually started throwing read errors last night so I need to replace that. 4 years 10 months ~ 41760 hours. da0 and da1 where warranted and da5 I just had to replace 9 months ago.

SMART Status Report Summary
Device Serial
Number SMART
Status Temp Power-On
Time Start/Stop
Count Spin
Retry
Count Realloc'd
Sectors Realloc
Events Current
Pending
Sectors Offline
Uncorrectable
Sectors UltraDMA
CRC
Errors Seek
Error
Health Last Test
Age (days) Last Test
Type
/dev/da0 WD-WCC7K0XNAULA PASSED 33*C 3y 0m 17d 10h 49 0 0 0 0 0 0 100% 0 Short
/dev/da1 WD-WCC4E6HZTD6X PASSED 35*C 2y 11m 9d 5h 50 0 0 0 0 0 0 100% 0 Short
/dev/da2 WD-WCC4E7SD0Z2F PASSED 34*C 4y 10m 22d 12h 74 0 0 0 0 0 0 100% 0 Short
/dev/da3 WD-WCC4E2YZNDX7 PASSED 36*C 4y 10m 22d 11h 72 0 0 0 0 0 0 100% 0 Short
/dev/da4 WD-WCC4E5KJ3Y0L PASSED 37*C 4y 10m 22d 11h 71 0 0 0 0 0 0 100% 0 Short
/dev/da5 WD-WCC7K1DF3RRY PASSED 34*C 0y 9m 16d 3h 9 0 0 0 0 0 0 100% 0 Short

@Wolfeman0101 , How do you manage presenting the report as you did?

Wolfeman0101 · Sep 1, 2020

Apollo said:
@Wolfeman0101 , How do you manage presenting the report as you did?

It's a script I run and have emailed to me every other day. Very handy. https://github.com/edgarsuit/FreeNAS-Report It also sends a config file every time. This is another good thread for scripts:

Scripts to report SMART, ZPool and UPS status, HDD/CPU T°, HDD identification and backup the config

I wanted to share my scripts because I think I'm not the only one who wants to monitor his FreeNAS server a bit more closely and in a more personalized way than the automatic nightly emails would do. Please note that you're the only one responsible of what you do in the CLI so don't blame me if...

www.ixsystems.com

dr_skorpio · Mar 20, 2022

I have a 6 drive zpool which had 2 Seagate and 4 WD Red drives, all 4Tb.

They are all in the 40-43k hour mark.

I’ve spent the last week bringing the system back to life after one of the SATA onboard controllers on the ASRock motherboard died AND one of the Seagate drives died.

After resilvering to a new WD Red Plus, I am now seeing faults on the older WD drives which are also likely to need replacing. These are around 5 years of run life so not too bad.

I’ll post exact models and run hours for those interested.

Apollo · Mar 20, 2022

dr_skorpio said:
I have a 6 drive zpool which had 2 Seagate and 4 WD Red drives, all 4Tb.

They are all in the 40-43k hour mark.

I’ve spent the last week bringing the system back to life after one of the SATA onboard controllers on the ASRock motherboard died AND one of the Seagate drives died.

After resilvering to a new WD Red Plus, I am now seeing faults on the older WD drives which are also likely to need replacing. These are around 5 years of run life so not too bad.

I’ll post exact models and run hours for those interested.

My setup based on 10 WD40EFRX pool is as follow:

WD-WCC7K******* 1y 1m 7d 12h
WD-WCC7K******* 2y 1m 20d 13h
WD-WCC7K******* 2y 7m 13d 1h
WD-WCC4E******* 5y 1m 19d 17h
WD-WCC4E******* 5y 1m 19d 17h
WD-WCC4E******* 5y 1m 19d 19h
WD-WCC4E******* 7y 6m 17d 0h
WD-WCC4E******* 7y 6m 17d 2h
WD-WCC4E******* 7y 8m 0d 0h
WD-WCC4E******* 7y 8m 0d 13h

fredmccellan · Apr 15, 2023

I have two WD Red Plus 4 TB drives in a NAS (Raid 0)) that are at 65,000 hours and 1 just failed. NAS is rebuilding as I write this.

drinking12many · May 1, 2023

my longest running 4TB RED is over 55000 hours, I have many other well over 30,000. So far I haven't had a single one die, I know its coming but so far it hasn't happened. I have only ever added new drives for more space since I built this server.

sretalla · May 1, 2023

I have a 6TB Red at 57035. I have lost something like 7 of what must be 10 of those I had originally though.

The other 2 remaining must be replacements for those which failed earlier as those only have 34 and 37K hours respectively.

Apollo · May 2, 2023

Update from about a year ago:

WD-WCC7Kxxx	17512
WD-WCC7Kxxx	19403
WD-WCC7Kxxx	28469
WD-WCC7Kxxx	32676
WD-WCC4Exxx	54736
WD-WCC4Exxx	54736
WD-WCC4Exxx	54738
WD-WCC4Exxx	75842
WD-WCC4Exxx	75843
WD-WCC4Exxx	76893
WD-WCC4Exxx	76906

NASbox · May 3, 2023

I've been experiencing a number of drives dropping off lately. This is the latest one to get kicked out of the pool during a scrub.
I haven't tested it yet, but when I tested the other two drives by running badblocks followed by a long self test, both badblocks and the self test completed without error..

I had one drive that I ran the badblocks/selftest, and then resilvered the drive..... It's been running for about 6 months. Given that a large part of the pool is used for a media library (some files haven't been touched since they were written over 5 years ago), is it possible that the drive surfaces need to be refreshed? Does the signal fade. I'm also wondering if case vibration might have been an issue. I've only ever seen these errors during scrub. While it is quite possible that scrub is the only time the bad spots get accessed, it is also the time the case experiences the most vibration since all 8 disk heads are very active during the scrub.

Thoughts anyone? The Smart Info and error logs are below FYI.

Code:

Model Family:     Western Digital Red
Device Model:     WDC WD60EFRX-68L0BN1

ID# ATTRIBUTE_NAME          FLAGS    VALUE WORST THRESH FAIL RAW_VALUE
  1 Raw_Read_Error_Rate     POSR-K   200   200   051    -    6
  3 Spin_Up_Time            POS--K   196   196   021    -    9183
  4 Start_Stop_Count        -O--CK   100   100   000    -    76
  5 Reallocated_Sector_Ct   PO--CK   200   200   140    -    0
  7 Seek_Error_Rate         -OSR-K   100   253   000    -    0
  9 Power_On_Hours          -O--CK   037   037   000    -    46097
 10 Spin_Retry_Count        -O--CK   100   253   000    -    0
 11 Calibration_Retry_Count -O--CK   100   253   000    -    0
 12 Power_Cycle_Count       -O--CK   100   100   000    -    76
192 Power-Off_Retract_Count -O--CK   200   200   000    -    74
193 Load_Cycle_Count        -O--CK   198   198   000    -    6071
194 Temperature_Celsius     -O---K   117   107   000    -    35
196 Reallocated_Event_Count -O--CK   200   200   000    -    0
197 Current_Pending_Sector  -O--CK   200   200   000    -    0
198 Offline_Uncorrectable   ----CK   100   253   000    -    0
199 UDMA_CRC_Error_Count    -O--CK   200   200   000    -    0
200 Multi_Zone_Error_Rate   ---R--   200   200   000    -    0

Error 6 [5] occurred at disk power-on lifetime: 46007 hours (1916 days + 23 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  40 -- 51 00 00 00 01 44 41 dc 08 40 00  Error: UNC at LBA = 0x14441dc08 = 5440134152

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  60 00 d8 00 38 00 01 44 41 e1 c8 40 00 45d+05:32:13.233  READ FPDMA QUEUED
  60 01 00 00 30 00 01 44 41 e0 c8 40 00 45d+05:32:13.233  READ FPDMA QUEUED
  60 01 00 00 28 00 01 44 41 df c8 40 00 45d+05:32:13.233  READ FPDMA QUEUED
  60 01 00 00 20 00 01 44 41 de c8 40 00 45d+05:32:13.233  READ FPDMA QUEUED
  60 01 00 00 18 00 01 44 41 dd c8 40 00 45d+05:32:13.233  READ FPDMA QUEUED

Error 5 [4] occurred at disk power-on lifetime: 46007 hours (1916 days + 23 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  40 -- 51 00 00 00 01 44 41 cf 78 40 00  Error: WP at LBA = 0x14441cf78 = 5440130936

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  61 00 10 00 40 00 02 ba a0 f0 90 40 00 45d+05:32:07.664  WRITE FPDMA QUEUED
  60 01 00 00 10 00 01 44 41 cc f8 40 00 45d+05:32:07.641  READ FPDMA QUEUED
  60 01 00 00 08 00 01 44 41 cb f8 40 00 45d+05:32:07.641  READ FPDMA QUEUED
  60 01 00 00 00 00 01 44 41 ca f8 40 00 45d+05:32:07.641  READ FPDMA QUEUED
  ea 00 00 00 00 00 00 00 00 00 00 40 00 45d+05:32:07.616  FLUSH CACHE EXT

Error 4 [3] occurred at disk power-on lifetime: 46007 hours (1916 days + 23 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  40 -- 51 00 00 00 01 44 41 c2 e0 40 00  Error: UNC at LBA = 0x14441c2e0 = 5440127712

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  60 00 10 00 70 00 02 ba a0 f2 90 40 00 45d+05:32:03.142  READ FPDMA QUEUED
  60 00 10 00 68 00 02 ba a0 f0 90 40 00 45d+05:32:03.142  READ FPDMA QUEUED
  60 00 10 00 60 00 00 00 40 02 90 40 00 45d+05:32:03.142  READ FPDMA QUEUED
  60 00 e8 00 58 00 01 44 41 c2 20 40 00 45d+05:32:03.142  READ FPDMA QUEUED
  60 01 00 00 50 00 01 44 41 c1 20 40 00 45d+05:32:03.142  READ FPDMA QUEUED

Error 3 [2] occurred at disk power-on lifetime: 46007 hours (1916 days + 23 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  40 -- 51 00 00 00 01 44 41 b6 50 40 00  Error: UNC at LBA = 0x14441b650 = 5440124496

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  60 00 10 00 88 00 02 ba a0 f2 90 40 00 45d+05:31:58.452  READ FPDMA QUEUED
  60 00 10 00 80 00 02 ba a0 f0 90 40 00 45d+05:31:58.452  READ FPDMA QUEUED
  60 00 10 00 78 00 00 00 40 02 90 40 00 45d+05:31:58.452  READ FPDMA QUEUED
  60 00 e8 00 70 00 01 44 41 c2 20 40 00 45d+05:31:58.452  READ FPDMA QUEUED
  60 01 00 00 68 00 01 44 41 c1 20 40 00 45d+05:31:58.452  READ FPDMA QUEUED

Error 2 [1] occurred at disk power-on lifetime: 46007 hours (1916 days + 23 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  40 -- 51 00 00 00 01 44 41 b4 00 40 00  Error: UNC at LBA = 0x14441b400 = 5440123904

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  60 00 10 00 b8 00 02 ba a0 f2 90 40 00 45d+05:31:53.695  READ FPDMA QUEUED
  60 00 10 00 b0 00 02 ba a0 f0 90 40 00 45d+05:31:53.695  READ FPDMA QUEUED
  60 00 10 00 a8 00 00 00 40 02 90 40 00 45d+05:31:53.695  READ FPDMA QUEUED
  60 00 e8 00 a0 00 01 44 41 c2 20 40 00 45d+05:31:53.695  READ FPDMA QUEUED
  60 01 00 00 98 00 01 44 41 c1 20 40 00 45d+05:31:53.695  READ FPDMA QUEUED

Error 1 [0] occurred at disk power-on lifetime: 46007 hours (1916 days + 23 hours)
  When the command that caused the error occurred, the device was active or idle.

  After command completion occurred, registers were:
  ER -- ST COUNT  LBA_48  LH LM LL DV DC
  -- -- -- == -- == == == -- -- -- -- --
  40 -- 51 00 00 00 01 44 41 a9 c0 40 00  Error: UNC at LBA = 0x14441a9c0 = 5440121280

  Commands leading to the command that caused the error were:
  CR FEATR COUNT  LBA_48  LH LM LL DV DC  Powered_Up_Time  Command/Feature_Name
  -- == -- == -- == == == -- -- -- -- --  ---------------  --------------------
  60 00 e8 00 a0 00 01 44 41 ba 38 40 00 45d+05:31:48.943  READ FPDMA QUEUED
  60 01 00 00 40 00 01 44 41 b9 38 40 00 45d+05:31:48.943  READ FPDMA QUEUED
  60 01 00 00 38 00 01 44 41 b8 38 40 00 45d+05:31:48.943  READ FPDMA QUEUED
  60 01 00 00 30 00 01 44 41 b7 38 40 00 45d+05:31:48.943  READ FPDMA QUEUED
  60 01 00 00 28 00 01 44 41 b6 38 40 00 45d+05:31:48.943  READ FPDMA QUEUED

Code:

Apr 29 09:35:20 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1459 loginfo 31080000
Apr 29 09:35:20 freenas mps0: (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 ad 70 00 00 01 00 00 00 
Apr 29 09:35:20 freenas Controller reported scsi ioc terminated tgt 11 SMID 1554 loginfo 31080000
Apr 29 09:35:20 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1363 loginfo 31080000
Apr 29 09:35:20 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1285 loginfo 31080000
Apr 29 09:35:20 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 726 loginfo 31080000
Apr 29 09:35:20 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1855 loginfo 31080000
Apr 29 09:35:20 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1902 loginfo 31080000
Apr 29 09:35:20 freenas mps0: (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:20 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 ae 70 00 00 01 00 00 00 
Apr 29 09:35:21 freenas Controller reported scsi ioc terminated tgt 11 SMID 546 loginfo 31080000
Apr 29 09:35:21 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 271 loginfo 31080000
Apr 29 09:35:21 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1559 loginfo 31080000
Apr 29 09:35:21 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1380 loginfo 31080000
Apr 29 09:35:21 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 241 loginfo 31080000
Apr 29 09:35:21 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1206 loginfo 31080000
Apr 29 09:35:21 freenas mps0: (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas Controller reported scsi ioc terminated tgt 11 SMID 1895 loginfo 31080000
Apr 29 09:35:21 freenas mps0: (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b3 38 00 00 01 00 00 00 
Apr 29 09:35:21 freenas Controller reported scsi ioc terminated tgt 11 SMID 1124 loginfo 31080000
Apr 29 09:35:21 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1161 loginfo 31080000
Apr 29 09:35:21 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 718 loginfo 31080000
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b4 38 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b5 38 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b6 38 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b7 38 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b8 38 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b9 38 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 aa 80 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 ab 80 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 ac 80 00 00 00 e8 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 af 70 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b0 70 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b1 70 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b2 70 00 00 00 c8 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 ba 38 00 00 00 e8 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 a9 80 00 00 01 00 00 00 
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): CAM status: SCSI Status Error
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): SCSI status: Check Condition
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): SCSI sense: MEDIUM ERROR asc:11,0 (Unrecovered read error)
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Info: 0x14441a9c0
Apr 29 09:35:21 freenas (da3:mps0:0:11:0): Error 5, Unretryable error
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 505 loginfo 31080000
Apr 29 09:35:25 freenas mps0: (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 bb 20 00 00 01 00 00 00 
Apr 29 09:35:25 freenas Controller reported scsi ioc terminated tgt 11 SMID 1989 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 2041 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 87 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 832 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1175 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1003 loginfo 31080000
Apr 29 09:35:25 freenas mps0: (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:25 freenas Controller reported scsi ioc terminated tgt 11 SMID 476 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1375 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 871 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 542 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1867 loginfo 31080000
Apr 29 09:35:25 freenas mps0: (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:25 freenas Controller reported scsi ioc terminated tgt 11 SMID 639 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1464 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1535 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 712 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1666 loginfo 31080000
Apr 29 09:35:25 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 359 loginfo 31080000
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 bc 20 00 00 01 00 00 00 
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 bd 20 00 00 01 00 00 00 
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 be 20 00 00 01 00 00 00 
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b4 38 00 00 01 00 00 00 
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b5 38 00 00 01 00 00 00 
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b6 38 00 00 01 00 00 00 
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b7 38 00 00 01 00 00 00 
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:25 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b8 38 00 00 01 00 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b9 38 00 00 01 00 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 ba 38 00 00 00 e8 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 bf 20 00 00 01 00 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 c0 20 00 00 01 00 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 c1 20 00 00 01 00 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 c2 20 00 00 00 e8 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(10). CDB: 28 00 00 40 02 90 00 00 10 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 02 ba a0 f0 90 00 00 00 10 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 02 ba a0 f2 90 00 00 00 10 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b3 38 00 00 01 00 00 00 
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): CAM status: SCSI Status Error
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): SCSI status: Check Condition
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): SCSI sense: MEDIUM ERROR asc:11,0 (Unrecovered read error)
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Info: 0x14441b400
Apr 29 09:35:26 freenas (da3:mps0:0:11:0): Error 5, Unretryable error
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 550 loginfo 31080000
Apr 29 09:35:30 freenas mps0: (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b7 38 00 00 01 00 00 00 
Apr 29 09:35:30 freenas Controller reported scsi ioc terminated tgt 11 SMID 1894 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 325 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 171 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 345 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 698 loginfo 31080000
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 1 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b8 38 00 00 01 00 00 00 
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 603 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 749 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 2086 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1466 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 395 loginfo 31080000
Apr 29 09:35:30 freenas mps0: (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 1 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b9 38 00 00 01 00 00 00 
Apr 29 09:35:30 freenas Controller reported scsi ioc terminated tgt 11 SMID 1204 loginfo 31080000
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 249 loginfo 31080000
Apr 29 09:35:30 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1509 loginfo 31080000
Apr 29 09:35:30 freenas mps0: (da3:mps0:0:11:0): Retrying command, 1 more tries remain
Apr 29 09:35:30 freenas Controller reported scsi ioc terminated tgt 11 SMID 2000 loginfo 31080000
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 ba 38 00 00 00 e8 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 1 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 bb 20 00 00 01 00 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 bc 20 00 00 01 00 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 bd 20 00 00 01 00 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 be 20 00 00 01 00 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 bf 20 00 00 01 00 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 c0 20 00 00 01 00 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 c1 20 00 00 01 00 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 c2 20 00 00 00 e8 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(10). CDB: 28 00 00 40 02 90 00 00 10 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 02 ba a0 f0 90 00 00 00 10 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 02 ba a0 f2 90 00 00 00 10 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Retrying command, 2 more tries remain
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 b6 38 00 00 01 00 00 00 
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): CAM status: SCSI Status Error
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): SCSI status: Check Condition
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): SCSI sense: MEDIUM ERROR asc:11,0 (Unrecovered read error)
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Info: 0x14441b650
Apr 29 09:35:30 freenas (da3:mps0:0:11:0): Error 5, Unretryable error
Apr 29 09:35:35 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1009 loginfo 31080000
Apr 29 09:35:35 freenas mps0: (da3:mps0:0:11:0): READ(10). CDB: 28 00 00 40 02 90 00 00 10 00 
Apr 29 09:35:35 freenas Controller reported scsi ioc terminated tgt 11 SMID 377 loginfo 31080000
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): Retrying command, 1 more tries remain
Apr 29 09:35:35 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1291 loginfo 31080000
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 02 ba a0 f0 90 00 00 00 10 00 00 
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): Retrying command, 1 more tries remain
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 02 ba a0 f2 90 00 00 00 10 00 00 
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): Retrying command, 1 more tries remain
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 c2 20 00 00 00 e8 00 00 
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): CAM status: SCSI Status Error
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): SCSI status: Check Condition
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): SCSI sense: MEDIUM ERROR asc:11,0 (Unrecovered read error)
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): Info: 0x14441c2e0
Apr 29 09:35:35 freenas (da3:mps0:0:11:0): Error 5, Unretryable error
Apr 29 09:35:39 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1528 loginfo 31080000
Apr 29 09:35:39 freenas mps0: (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 cf f8 00 00 01 00 00 00 
Apr 29 09:35:39 freenas Controller reported scsi ioc terminated tgt 11 SMID 158 loginfo 31080000
Apr 29 09:35:39 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 324 loginfo 31080000
Apr 29 09:35:39 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 239 loginfo 31080000
Apr 29 09:35:39 freenas mps0: (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 d0 f8 00 00 01 00 00 00 
Apr 29 09:35:39 freenas Controller reported scsi ioc terminated tgt 11 SMID 1563 loginfo 31080000
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 d1 f8 00 00 00 b8 00 00 
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): WRITE(16). CDB: 8a 00 00 00 00 02 ba a0 f0 90 00 00 00 10 00 00 
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): WRITE(16). CDB: 8a 00 00 00 00 02 ba a0 f2 90 00 00 00 10 00 00 
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 ce f8 00 00 01 00 00 00 
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): CAM status: SCSI Status Error
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): SCSI status: Check Condition
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): SCSI sense: MEDIUM ERROR asc:11,0 (Unrecovered read error)
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): Info: 0x14441cf78
Apr 29 09:35:39 freenas (da3:mps0:0:11:0): Error 5, Unretryable error
Apr 29 09:35:45 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 721 loginfo 31080000
Apr 29 09:35:45 freenas mps0: (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 dc c8 00 00 01 00 00 00 
Apr 29 09:35:45 freenas Controller reported scsi ioc terminated tgt 11 SMID 262 loginfo 31080000
Apr 29 09:35:45 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 215 loginfo 31080000
Apr 29 09:35:45 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 1365 loginfo 31080000
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 dd c8 00 00 01 00 00 00 
Apr 29 09:35:45 freenas mps0: Controller reported scsi ioc terminated tgt 11 SMID 95 loginfo 31080000
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 de c8 00 00 01 00 00 00 
Apr 29 09:35:45 freenas mps0: (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:45 freenas Controller reported scsi ioc terminated tgt 11 SMID 2107 loginfo 31080000
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 df c8 00 00 01 00 00 00 
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 e0 c8 00 00 01 00 00 00 
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 e1 c8 00 00 00 d8 00 00 
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): CAM status: CCB request completed with an error
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): Retrying command, 3 more tries remain
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): READ(16). CDB: 88 00 00 00 00 01 44 41 db c8 00 00 01 00 00 00 
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): CAM status: SCSI Status Error
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): SCSI status: Check Condition
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): SCSI sense: MEDIUM ERROR asc:11,0 (Unrecovered read error)
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): Info: 0x14441dc08
Apr 29 09:35:45 freenas (da3:mps0:0:11:0): Error 5, Unretryable error

Important Announcement for the TrueNAS Community.

WD Red (WD60EFRX-68M) life seems to be about 40,000 Hours (What is your experience?)

NASbox

Guru

Evertb1

Guru

NASbox

Guru

Evertb1

Guru

Wolfeman0101

Patron

NASbox

Guru

Wolfeman0101

Patron

NASbox

Guru

Wolfeman0101

Patron

Apollo

Wizard

Wolfeman0101

Patron

Scripts to report SMART, ZPool and UPS status, HDD/CPU T°, HDD identification and backup the config

dr_skorpio

Cadet

Apollo

Wizard

fredmccellan

Cadet

drinking12many

Contributor

sretalla

Powered by Neutrality

Apollo

Wizard

NASbox

Guru

Similar threads