One or more devices are faulted in response to IO failures.

Status
Not open for further replies.

Amsoil_Jim

Contributor
Joined
Feb 22, 2016
Messages
175
Motherboard: X9DRI-LN4F+
CPU: Intel Xeon E5 2690v1 2.9Ghz x2
Memory: 192GB DDR3 SDRAM ECC (24x8GB)
Disks: 6 x WD Red 2TB Raidz1 & 1 x 128GB SSD stripe
Boot Disk: 1 x SanDisk Ultra Fit 64GB
Raid Controller: LSI 9211-4I HBA
Back Plane: SAS2-846EL1

I received this email from my machine:
Code:
Checking status of zfs pools:
NAME		   SIZE  ALLOC   FREE  EXPANDSZ   FRAG	CAP  DEDUP  HEALTH  ALTROOT
Media		 16.2T  5.06T  11.2T		 -	 6%	31%  1.00x  ONLINE  /mnt
VM_Disk		117G  10.2G   107G		 -	 4%	 8%  1.00x  ONLINE  /mnt
freenas-boot  57.5G  2.80G  54.7G		 -	  -	 4%  1.00x  ONLINE  -

  pool: freenas-boot
 state: ONLINE
status: One or more devices are faulted in response to IO failures.
action: Make sure the affected devices are connected, then run 'zpool clear'.
   see: http://illumos.org/msg/ZFS-8000-HC
  scan: scrub in progress since Sun Jul  9 03:45:00 2017
		1.82G scanned out of 2.79G at 22.8K/s, 12h26m to go
		0 repaired, 65.16% done
config:

		NAME		STATE	 READ WRITE CKSUM
		freenas-boot  ONLINE	   1	10	 0
		  da6p2	 ONLINE	   0	32	 0

errors: 3 data errors, use '-v' for a list

Local system status:
 3:01AM  up 5 days, 11:55, 0 users, load averages: 0.01, 0.08, 0.07

-- End of daily output --

and then this one
Code:
cannot get history for 'freenas-boot': pool I/O is currently suspended
cannot get history for 'freenas-boot': pool I/O is currently suspended
   skipping scrubbing of pool 'freenas-boot':
	  can't get last scrubbing date


Once I got home I tried to access the GUI and it would not load, I tried to access via SSH and got no response.
All my jails were still running and accessible.
So I reset the machine and now it says there are no errors.
Code:
root@FreeNAS:~ # zpool status -v

  pool: Media

 state: ONLINE

  scan: scrub repaired 0 in 3h55m with 0 errors on Sun Jul  2 04:30:27 2017

config:


	NAME											STATE	 READ WRITE CKSUM

	Media										   ONLINE	   0	 0	 0

	  raidz1-0									  ONLINE	   0	 0	 0

		gptid/6817d49b-dd80-11e5-96c2-d05099c00c66  ONLINE	   0	 0	 0

		gptid/68fb7018-dd80-11e5-96c2-d05099c00c66  ONLINE	   0	 0	 0

		gptid/69dbbaca-dd80-11e5-96c2-d05099c00c66  ONLINE	   0	 0	 0

		gptid/6ab7c822-dd80-11e5-96c2-d05099c00c66  ONLINE	   0	 0	 0

		gptid/6b99a30e-dd80-11e5-96c2-d05099c00c66  ONLINE	   0	 0	 0

		gptid/6c7aadf1-dd80-11e5-96c2-d05099c00c66  ONLINE	   0	 0	 0


errors: No known data errors


  pool: VM_Disk

 state: ONLINE

  scan: none requested

config:


	NAME										  STATE	 READ WRITE CKSUM

	VM_Disk									   ONLINE	   0	 0	 0

	  gptid/86f788f5-628b-11e7-8406-002590826ed0  ONLINE	   0	 0	 0


errors: No known data errors


  pool: freenas-boot

 state: ONLINE

  scan: scrub repaired 0 in 35h53m with 0 errors on Mon Jul 10 15:38:32 2017

config:


	NAME		STATE	 READ WRITE CKSUM

	freenas-boot  ONLINE	   0	 0	 0

	  da6p2	 ONLINE	   0	 0	 0


errors: No known data errors


So I'm a little confused, some thing happened, I was unable to access the interface, and it generated the email.
But now after the restart it appears to be fine.
Is my flash drive going? it is less than a month old.

camcontrol devlist shows the following
Code:
root@FreeNAS:~ # camcontrol devlist

<ADATA SU800 Q0125A>			   at scbus2 target 0 lun 0 (pass0,ada0)

<LSI SAS2X36 0e0b>				 at scbus7 target 8 lun 0 (pass1,ses0)

<ATA WDC WD30EFRX-68E 0A82>		at scbus7 target 10 lun 0 (pass2,da0)

<ATA WDC WD30EFRX-68E 0A82>		at scbus7 target 11 lun 0 (pass3,da1)

<ATA WDC WD30EFRX-68E 0A82>		at scbus7 target 12 lun 0 (pass4,da2)

<ATA WDC WD30EFRX-68E 0A82>		at scbus7 target 13 lun 0 (pass5,da3)

<ATA WDC WD30EFRX-68E 0A82>		at scbus7 target 14 lun 0 (pass6,da4)

<ATA WDC WD30EFRX-68E 0A82>		at scbus7 target 15 lun 0 (pass7,da5)

<SanDisk Ultra Fit 1.00>		   at scbus9 target 0 lun 0 (pass8,da6)
 

rs225

Guru
Joined
Jun 28, 2014
Messages
878
Is the .system dataset on the USB?

In any case, make sure you have a backup of the config, because you either had a glitch, or the USB is going.
 

Amsoil_Jim

Contributor
Joined
Feb 22, 2016
Messages
175
Is the .system dataset on the USB?

In any case, make sure you have a backup of the config, because you either had a glitch, or the USB is going.
the USB is only the boot drive.
I have a backup of the config emailed to me several time a month so I should be good there.
I also scrubbed the USB drive again just to make sure and received no errors again, so I guess for the moment it's wait and see then.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,419
If you're going to use a USB flash drive as boot you should use two in mirror. Check the manual for how to add another boot mirror.

And make sure the system data set is not on the boot drive.
 

Amsoil_Jim

Contributor
Joined
Feb 22, 2016
Messages
175
@ 10pm last night this shows up
Code:
(da6:umass-sim0:0:0:0): READ(10). CDB: 28 00 00 24 0c 4a 00 00 47 00 

(da6:umass-sim0:0:0:0): CAM status: CCB request completed with an error

(da6:umass-sim0:0:0:0): Retrying command

(da6:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 02 30 a5 52 00 00 08 00 

(da6:umass-sim0:0:0:0): CAM status: CCB request completed with an error

(da6:umass-sim0:0:0:0): Retrying command

(da6:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 02 30 a5 52 00 00 08 00 

(da6:umass-sim0:0:0:0): CAM status: CCB request completed with an error

(da6:umass-sim0:0:0:0): Retrying command

(da6:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 02 30 a5 52 00 00 08 00 

(da6:umass-sim0:0:0:0): CAM status: CCB request completed with an error

(da6:umass-sim0:0:0:0): Retrying command

(da6:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 02 30 a5 52 00 00 08 00 

(da6:umass-sim0:0:0:0): CAM status: CCB request completed with an error

(da6:umass-sim0:0:0:0): Retrying command

(da6:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 02 30 a5 52 00 00 08 00 

(da6:umass-sim0:0:0:0): CAM status: CCB request completed with an error

(da6:umass-sim0:0:0:0): Error 5, Retries exhausted

(da6:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 02 30 a6 bb 00 00 0d 00 

(da6:umass-sim0:0:0:0): CAM status: CCB request completed with an error

(da6:umass-sim0:0:0:0): Retrying command

(da6:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 02 30 a6 bb 00 00 0d 00 

(da6:umass-sim0:0:0:0): CAM status: CCB request completed with an error

(da6:umass-sim0:0:0:0): Retrying command

 
Status
Not open for further replies.
Top