MrWraith
Dabbler
- Joined
- Jul 19, 2016
- Messages
- 10
Hi guys. I've done a lot of searching on this, and followed some advice found here, but it has got to the stage where I need to just ask for help.
What I've done so far:
I have a FreeNAS box (HP ProLiant N40L) with four WD 3TB drives. (WD-WCAWZXXXXXXX).
The day before yesterday, I had an error "CRITICAL: Device: /dev/ada3, 1 Currently unreadable (pending) sector"
I did some reading, learned about SMART long tests, and did one of those. It took ~8 hours, and after that I had 4 Currently unreadable (pending) sectors.
I tried to write to those bad sectors using dd (as per suggestions on these forums). dd gave me an input/output error when I tried to use it. I didn't know if that meant it was doing nothing, or if the error was a sign that the block was being reassigned. The LBA_of_first_error was changing after my attempted dd and new long self-test, so I thought it may be working. However, after a reboot and another long test, I still had 4 unreadable (pending) sectors.
So I did a zpool scrub (as per some more advice on this forum). It took 18hrs, still 4 unreadable sectors, and now more problems. I have a FreeNAS GUI error "The volume VolumeZero (ZFS) state is ONLINE: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected." (same error in my zpool status output).
I just noticed some errors in dmesg when preparing to write this thread. I'll attach the output below.
Where I am now:
These disks are relatively old, and have been running without spinning down for years. I will replace one if I need to, but I'd like to explore other options first if other options exist. Disks are expensive in Australia and I'm a student.
I've already externally backed up all the critical info to my unRAID box, and i'm currently backing up the rest (apart from the 6TB of movies that I can just re-download in case of total failure).
Now for the info dumps:
DMESG Output:
(not sure if the whole thing was needed. Errors near the bottom)
zpool status
smartctl -a /dev/ada3
What I've done so far:
I have a FreeNAS box (HP ProLiant N40L) with four WD 3TB drives. (WD-WCAWZXXXXXXX).
The day before yesterday, I had an error "CRITICAL: Device: /dev/ada3, 1 Currently unreadable (pending) sector"
I did some reading, learned about SMART long tests, and did one of those. It took ~8 hours, and after that I had 4 Currently unreadable (pending) sectors.
I tried to write to those bad sectors using dd (as per suggestions on these forums). dd gave me an input/output error when I tried to use it. I didn't know if that meant it was doing nothing, or if the error was a sign that the block was being reassigned. The LBA_of_first_error was changing after my attempted dd and new long self-test, so I thought it may be working. However, after a reboot and another long test, I still had 4 unreadable (pending) sectors.
So I did a zpool scrub (as per some more advice on this forum). It took 18hrs, still 4 unreadable sectors, and now more problems. I have a FreeNAS GUI error "The volume VolumeZero (ZFS) state is ONLINE: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected." (same error in my zpool status output).
I just noticed some errors in dmesg when preparing to write this thread. I'll attach the output below.
Where I am now:
These disks are relatively old, and have been running without spinning down for years. I will replace one if I need to, but I'd like to explore other options first if other options exist. Disks are expensive in Australia and I'm a student.
I've already externally backed up all the critical info to my unRAID box, and i'm currently backing up the rest (apart from the 6TB of movies that I can just re-download in case of total failure).
Now for the info dumps:
DMESG Output:
(not sure if the whole thing was needed. Errors near the bottom)
Code:
[root@Micro] ~# dmesg tun0: link state changed to DOWN tun0: link state changed to UP arp: 192.168.1.66 moved from 02:61:7a:00:0c:0a to e8:39:35:ee:28:49 on epair1b tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP arp: 192.168.1.66 moved from 02:61:7a:00:0c:0a to e8:39:35:ee:28:49 on epair1b tun0: link state changed to DOWN tun0: link state changed to UP epair0a: link state changed to DOWN epair0b: link state changed to DOWN ifa_del_loopback_route: deletion failed Freed UMA keg (udp_inpcb) was not empty (40 items). Lost 4 pages of memory. Freed UMA keg (udpcb) was not empty (336 items). Lost 2 pages of memory. Freed UMA keg (tcptw) was not empty (300 items). Lost 6 pages of memory. Freed UMA keg (tcp_inpcb) was not empty (160 items). Lost 16 pages of memory. Freed UMA keg (sackhole) was not empty (101 items). Lost 1 pages of memory. Freed UMA keg (tcpcb) was not empty (44 items). Lost 11 pages of memory. Freed UMA keg (ripcb) was not empty (20 items). Lost 2 pages of memory. hhook_vnet_uninit: hhook_head type=1, id=1 cleanup required hhook_vnet_uninit: hhook_head type=1, id=0 cleanup required epair1a: link state changed to DOWN epair1b: link state changed to DOWN bridge0: link state changed to DOWN bge0: promiscuous mode disabled tun0: link state changed to DOWN pid 1608 (syslog-ng), uid 0: exited on signal 6 (core dumped) Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...0 0 0 0 0 0 0 done All buffers synced. GEOM_ELI: Device ada0p1.eli destroyed. GEOM_ELI: Detached ada0p1.eli on last close. GEOM_ELI: Device ada1p1.eli destroyed. GEOM_ELI: Detached ada1p1.eli on last close. GEOM_ELI: Device ada2p1.eli destroyed. GEOM_ELI: Detached ada2p1.eli on last close. GEOM_ELI: Device ada3p1.eli destroyed. GEOM_ELI: Detached ada3p1.eli on last close. Uptime: 66d20h33m11s Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.3-RELEASE-p26 #0 r281084+93c5885: Mon Sep 28 13:25:20 PDT 2015 root@build3.ixsystems.com:/tank/home/stable-builds/FN/objs/os-base/amd64/tank/home/stable-builds/FN/FreeBSD/src/sys/FREENAS.amd64 amd64 gcc version 4.2.1 20070831 patched [FreeBSD] CPU: AMD Turion(tm) II Neo N40L Dual-Core Processor (1497.54-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f63 Family = 0x10 Model = 0x6 Stepping = 3 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0x802009<SSE3,MON,CX16,POPCNT> AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!> AMD Features2=0x837ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,NodeId> TSC: P-state invariant real memory = 9126805504 (8704 MB) avail memory = 8104435712 (7728 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: <HP ProLiant> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 WARNING: VIMAGE (virtualized network stack) is a highly experimental feature. ioapic0 <Version 2.1> irqs 0-23 on motherboard ispfw: registered firmware <isp_1040> ispfw: registered firmware <isp_1040_it> ispfw: registered firmware <isp_1080> ispfw: registered firmware <isp_1080_it> ispfw: registered firmware <isp_12160> ispfw: registered firmware <isp_12160_it> ispfw: registered firmware <isp_2100> ispfw: registered firmware <isp_2200> ispfw: registered firmware <isp_2300> ispfw: registered firmware <isp_2322> ispfw: registered firmware <isp_2400> ispfw: registered firmware <isp_2400_multi> ispfw: registered firmware <isp_2500> ispfw: registered firmware <isp_2500_multi> kbd1 at kbdmux0 cryptosoft0: <software crypto> on motherboard aesni0: No AESNI support. padlock0: No ACE support. acpi0: <HP ProLiant> on motherboard acpi0: Power Button (fixed) acpi0: reservation of fee00000, 1000 (3) failed acpi0: reservation of ffb80000, 80000 (3) failed acpi0: reservation of fec10000, 20 (3) failed acpi0: reservation of fed80000, 1000 (3) failed acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, d7f00000 (3) failed cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 550 Event timer "HPET1" frequency 14318180 Hz quality 450 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0 pci1: <ACPI PCI bus> on pcib1 vgapci0: <VGA-compatible display> port 0xe000-0xe0ff mem 0xf0000000-0xf7ffffff,0xfe8f0000-0xfe8fffff,0xfe700000-0xfe7fffff irq 18 at device 5.0 on pci1 vgapci0: Boot video device pcib2: <ACPI PCI-PCI bridge> irq 18 at device 6.0 on pci0 pci2: <ACPI PCI bus> on pcib2 bge0: <HP NC107i PCIe Gigabit Server Adapter, ASIC rev. 0x5784100> mem 0xfe9f0000-0xfe9fffff irq 18 at device 0.0 on pci2 bge0: CHIP ID 0x05784100; ASIC REV 0x5784; CHIP REV 0x57841; PCI-E miibus0: <MII bus> on bge0 brgphy0: <BCM5784 10/100/1000baseT PHY> PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: e8:39:35:ee:28:49 ahci0: <AMD SB7x0/SB8x0/SB9x0 AHCI SATA controller> port 0xd000-0xd007,0xc000-0xc003,0xb000-0xb007,0xa000-0xa003,0x9000-0x900f mem 0xfe6ffc00-0xfe6fffff irq 19 at device 17.0 on pci0 ahci0: AHCI v1.20 with 4 3Gbps ports, Port Multiplier supported ahcich0: <AHCI channel> at channel 0 on ahci0 ahcich1: <AHCI channel> at channel 1 on ahci0 ahcich2: <AHCI channel> at channel 2 on ahci0 ahcich3: <AHCI channel> at channel 3 on ahci0 ohci0: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfe6fe000-0xfe6fefff irq 18 at device 18.0 on pci0 usbus0 on ohci0 ehci0: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfe6ff800-0xfe6ff8ff irq 17 at device 18.2 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci0 ohci1: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfe6fd000-0xfe6fdfff irq 18 at device 19.0 on pci0 usbus2 on ohci1 ehci1: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfe6ff400-0xfe6ff4ff irq 17 at device 19.2 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci1 atapci0: <ATI IXP700/800 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xff00-0xff0f at device 20.1 on pci0 ata0: <ATA channel> at channel 0 on atapci0 ata1: <ATA channel> at channel 1 on atapci0 isab0: <PCI-ISA bridge> at device 20.3 on pci0 isa0: <ISA bus> on isab0 pcib3: <ACPI PCI-PCI bridge> at device 20.4 on pci0 pci3: <ACPI PCI bus> on pcib3 ohci2: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfe6fc000-0xfe6fcfff irq 18 at device 22.0 on pci0 usbus4 on ohci2 ehci2: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfe6ff000-0xfe6ff0ff irq 17 at device 22.2 on pci0 usbus5: EHCI version 1.0 usbus5 on ehci2 amdtemp0: <AMD CPU On-Die Thermal Sensors> on hostb4 acpi_button0: <Power Button> on acpi0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 wbwd0: HEFRAS and EFER do not align: EFER 0x2e DevID 0xff DevRev 0xff CR26 0xff hwpstate0: <Cool`n'Quiet 2.0> on cpu0 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled usbus0: 12Mbps Full Speed USB v1.0 usbus1: 480Mbps High Speed USB v2.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 usbus4: 12Mbps Full Speed USB v1.0 usbus5: 480Mbps High Speed USB v2.0 ugen0.1: <ATI> at usbus0 uhub0: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 ugen1.1: <ATI> at usbus1 uhub1: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 ugen2.1: <ATI> at usbus2 uhub2: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2 ugen3.1: <ATI> at usbus3 uhub3: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3 ugen4.1: <ATI> at usbus4 uhub4: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4 ugen5.1: <ATI> at usbus5 uhub5: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus5 uhub4: 4 ports with 4 removable, self powered uhub0: 5 ports with 5 removable, self powered uhub2: 5 ports with 5 removable, self powered uhub5: 4 ports with 4 removable, self powered uhub1: 5 ports with 5 removable, self powered uhub3: 5 ports with 5 removable, self powered ugen3.2: <Kingston> at usbus3 umass0: <Kingston DataTraveler 3.0, class 0/0, rev 2.10/1.00, addr 2> on usbus3 umass0: SCSI over Bulk-Only; quirks = 0x8100 umass0:7:0:-1: Attached to scbus7 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: <WDC WD30EZRX-00MMMB0 80.00A80> ATA-8 SATA 3.x device ada0: Serial Number WD-WCAWZ2042347 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C) ada0: quirks=0x1<4K> ada0: Previously was known as ad4 da0 at umass-sim0 bus 0 scbus7 target 0 lun 0 da0: <Kingston DataTraveler 3.0 PMAP> Removable Direct Access SCSI-6 device da0: Serial Number 60A44C4250DBBD904B47169E da0: 40.000MB/s transfers da0: 7500MB (15360000 512 byte sectors: 255H 63S/T 956C) da0: quirks=0x2<NO_6_BYTE> ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: <WDC WD30EZRX-00MMMB0 80.00A80> ATA-8 SATA 3.x device ada1: Serial Number WD-WCAWZ2023116 ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus2 target 0 lun 0 ada2: <WDC WD30EZRX-00MMMB0 80.00A80> ATA-8 SATA 3.x device ada2: Serial Number WD-WMAWZ0178959 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C) ada2: quirks=0x1<4K> ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus3 target 0 lun 0 ada3: <WDC WD30EZRX-00MMMB0 80.00A80> ATA-8 SATA 3.x device ada3: Serial Number WD-WMAWZ0183668 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: Command Queueing enabled ada3: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C) ada3: quirks=0x1<4K> ada3: Previously was known as ad10 SMP: AP CPU #1 Launched! Timecounter "TSC" frequency 1497538359 Hz quality 800 Trying to mount root from zfs:freenas-boot/ROOT/FreeNAS-9.3-STABLE-201509282017 []... GEOM_RAID5: Module loaded, version 1.3.20140711.62 (rev f91e28e40bf7) ipmi0: <IPMI System Interface> on isa0 ipmi0: KCS mode found at mem 0x0 alignment 0x1 on isa ipmi0: couldn't configure I/O resource device_attach: ipmi0 attach returned 6 wbwd0: HEFRAS and EFER do not align: EFER 0x2e DevID 0xff DevRev 0xff CR26 0xff GEOM_ELI: Device ada0p1.eli created. GEOM_ELI: Encryption: AES-XTS 256 GEOM_ELI: Crypto: software GEOM_ELI: Device ada1p1.eli created. GEOM_ELI: Encryption: AES-XTS 256 GEOM_ELI: Crypto: software GEOM_ELI: Device ada2p1.eli created. GEOM_ELI: Encryption: AES-XTS 256 GEOM_ELI: Crypto: software GEOM_ELI: Device ada3p1.eli created. GEOM_ELI: Encryption: AES-XTS 256 GEOM_ELI: Crypto: software vboxdrv: fAsync=0 offMin=0x358 offMax=0xb5a bridge0: Ethernet address: 02:e9:08:1c:c8:00 bge0: promiscuous mode enabled bridge0: link state changed to UP epair0a: Ethernet address: 02:b4:99:00:0b:0a epair0b: Ethernet address: 02:b4:99:00:0c:0b epair0a: link state changed to UP epair0b: link state changed to UP epair0a: promiscuous mode enabled ng_ether_ifnet_arrival_event: can't re-name node epair0b epair1a: Ethernet address: 02:35:32:00:0c:0a epair1b: Ethernet address: 02:35:32:00:0d:0b epair1a: link state changed to UP epair1b: link state changed to UP epair1a: promiscuous mode enabled ng_ether_ifnet_arrival_event: can't re-name node epair1b tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP arp: 192.168.1.66 moved from 02:35:32:00:0c:0a to e8:39:35:ee:28:49 on epair1b (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 a8 b1 b5 40 f5 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 50 b2 b5 40 f5 00 00 00 00 (ada3:ahcich3:0:0:0): Retrying command (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 a8 b1 b5 40 f5 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 01 (ILI) (ada3:ahcich3:0:0:0): RES: 41 01 50 b2 b5 40 f5 00 00 00 00 (ada3:ahcich3:0:0:0): Retrying command (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 a8 b1 b5 40 f5 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 50 b2 b5 40 f5 00 00 00 00 (ada3:ahcich3:0:0:0): Retrying command (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 a8 b1 b5 40 f5 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 50 b2 b5 40 f5 00 00 00 00 (ada3:ahcich3:0:0:0): Retrying command (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 a8 b1 b5 40 f5 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 50 b2 b5 40 f5 00 00 00 00 (ada3:ahcich3:0:0:0): Error 5, Retries exhausted (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 30 f3 99 40 f7 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 80 f3 99 40 f7 00 00 00 00 (ada3:ahcich3:0:0:0): Retrying command (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 30 f3 99 40 f7 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 80 f3 99 40 f7 00 00 00 00 (ada3:ahcich3:0:0:0): Retrying command (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 30 f3 99 40 f7 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 80 f3 99 40 f7 00 00 00 00 (ada3:ahcich3:0:0:0): Retrying command (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 30 f3 99 40 f7 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 80 f3 99 40 f7 00 00 00 00 (ada3:ahcich3:0:0:0): Retrying command (ada3:ahcich3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 30 f3 99 40 f7 00 00 01 00 00 (ada3:ahcich3:0:0:0): CAM status: ATA Status Error (ada3:ahcich3:0:0:0): ATA status: 41 (DRDY ERR), error: 40 (UNC ) (ada3:ahcich3:0:0:0): RES: 41 40 80 f3 99 40 f7 00 00 00 00 (ada3:ahcich3:0:0:0): Error 5, Retries exhausted tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP arp: 192.168.1.66 moved from 02:35:32:00:0c:0a to e8:39:35:ee:28:49 on epair1b
zpool status
Code:
[root@Micro] ~# zpool status pool: VolumeZero state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: http://illumos.org/msg/ZFS-8000-9P scan: scrub repaired 284K in 18h7m with 0 errors on Wed Jul 20 07:39:31 2016 config: NAME STATE READ WRITE CKSUM VolumeZero ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 gptid/6b5bbd30-abaf-11e1-9c11-e83935ee2849 ONLINE 0 0 0 gptid/6c218b94-abaf-11e1-9c11-e83935ee2849 ONLINE 0 0 0 gptid/6cf0180d-abaf-11e1-9c11-e83935ee2849 ONLINE 0 0 0 gptid/6db64f2c-abaf-11e1-9c11-e83935ee2849 ONLINE 0 0 1 errors: No known data errors pool: freenas-boot state: ONLINE status: One or more devices are configured to use a non-native block size. Expect reduced performance. action: Replace affected devices with devices that support the configured block size, or migrate data to a properly configured pool. scan: scrub repaired 0 in 0h1m with 0 errors on Thu Jun 16 03:46:22 2016 config: NAME STATE READ WRITE CKSUM freenas-boot ONLINE 0 0 0 da0p2 ONLINE 0 0 0 block size: 8192B configured, 8388608B native errors: No known data errors
smartctl -a /dev/ada3
Code:
[root@Micro] ~# smartctl -a /dev/ada3 smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p26 amd64] (local build) Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Western Digital Green Device Model: WDC WD30EZRX-00MMMB0 Serial Number: WD-WMAWZ0183668 LU WWN Device Id: 5 0014ee 206df0349 Firmware Version: 80.00A80 User Capacity: 3,000,592,982,016 bytes [3.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Device is: In smartctl database [for details use: -P show] ATA Version is: ATA8-ACS (minor revision not indicated) SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Wed Jul 20 11:22:49 2016 AEST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x84) Offline data collection activity was suspended by an interrupting command from host. Auto Offline Data Collection: Enabled. Self-test execution status: ( 121) The previous self-test completed having the read element of the test failed. Total time to complete Offline data collection: (51660) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 496) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x3035) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 151 150 021 Pre-fail Always - 9408 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 64 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 056 056 000 Old_age Always - 32747 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 62 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 48 193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 3002349 194 Temperature_Celsius 0x0022 116 106 000 Old_age Always - 36 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 4 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 2 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed: read failure 90% 32725 3825966784 # 2 Extended offline Completed: read failure 90% 32725 3825966788 # 3 Extended offline Completed: read failure 90% 32725 3825966790 # 4 Extended offline Completed: read failure 90% 32725 3825966786 # 5 Extended offline Completed: read failure 90% 32725 3825966788 # 6 Extended offline Completed: read failure 90% 32725 3825966790 # 7 Extended offline Completed: read failure 70% 32715 1987001840 SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.