ZeroGravity
Cadet
- Joined
- Apr 17, 2020
- Messages
- 1
I will start with some background first...
I was on FREENAS-11.2 RELEASE (Build Date Dec 5, 2018 21:28) for a long while, with no issues... But I decided to update to the latest after talking to a friend while we were playing D&D.
I made a back up of my config, then clicked to update to the latest in the 11.2 train (11.2-U8), at first everything looked fine.. I was browsing in another monitor and hit F5 top refresh that one, and it refreshed the Freenas window instead. I lost my progress bar, so I waited a long while, nothing was happening so I poked around a bit and decided to restart (I now know that this was a stupid move).
My FREENAS then refused to boot, so I went to the monitor that is connected to my box and loaded the 11.2 version that had worked forever for me without issue.. The system booted, and I figured I would try again. Again, I selected to update, the progress bar popped up... and about 5 min later, I got an email telling my my pool was in a degraded state. Afraid of leaving the window this time, I decided to wait until morning to see what had gone wrong, and went to bed, letting it finish the update.
I look in the morning, and it appears as though the update completed successfully, except when I click on anything in the Storage section on the left, nothing happens.. If I right click and open in new tab, it works fine, and will continue to work fine in that new window.
Now here is where things get interesting.. I open pool status, and it shows my pool status as DEGRADED, RaidZ2 Status as DEGRADED, and where it would normally say ada7p2, it now says "/dev/gptid/0e33a067-6b9e-11e4-a9c1-40167e65b1e6", and the status is UNAVAIL.
When I try to bring it online, it loads for a moment, and then stays on UNAVAIL.
When I try to bring it offline, it changes to OFFLINE.
In my search, I had found others had to resilver the drive when they had a hard drive fail, and it shows UNAVAIL, or when they accidentally put a boot partition on their drive, so I thought, okay, I wonder what will happen if I try to resilver the drive? But when I click on replace in the drop down, click the force check box, there are no options in the drop down box. it just says Member Disk*, and the REPLACE DISK button is greyed out.
I have also tried reseating the cables on the drive, and then even swapped the drive with another in the pool to see if it was a bad cable or something. No changes at all to the above.
I realize this could be a failed drive, but the timing just seems to coincidental that it would happen right after a failed update. At least to me it does.... Its also worth noting that I can see the drive in the bios, and it looks fine as well.
On top of all this, every once in a while now, I am getting booted from the web gui. It looks like the link status goes to down, and then comes back up on its own. This too only occurred after the failed update, but is kind of the least of my worries right now.
My hardware:
Motherboard: Asus P8B75-M
CPU: i3-2100 3.10 Ghz
RAM: 32GB (4 x 8GB)
Hard Drives: 8 x 4TB WD Black - Model: WD4003FZEX
Hard Disk Controller Card: Vantec 2 Channel Sata PCie Host Card UGT-ST622
Network Card: Just using the onboard nic
When I first boot up:
When I click on OFFLINE:
When I click on ONLINE (and it changes to UNAVAIL):
Sorry for the long post, I tried to be as detailed as I could and not leave out anything I thought would be helpful.
I have done my best to look for answers myself, I'm just at a loss on what could be causing this at this point. I am hoping it is something incredibly simple that I have just overlooked *fingers crossed*. Hoping someone can point me in the right direction!
Lastly, I apologize if I have posted this in the wrong area.. I know I have a few issues here, but the the degraded pool seems to be the worst of it.
Thank you!!
I was on FREENAS-11.2 RELEASE (Build Date Dec 5, 2018 21:28) for a long while, with no issues... But I decided to update to the latest after talking to a friend while we were playing D&D.
I made a back up of my config, then clicked to update to the latest in the 11.2 train (11.2-U8), at first everything looked fine.. I was browsing in another monitor and hit F5 top refresh that one, and it refreshed the Freenas window instead. I lost my progress bar, so I waited a long while, nothing was happening so I poked around a bit and decided to restart (I now know that this was a stupid move).
My FREENAS then refused to boot, so I went to the monitor that is connected to my box and loaded the 11.2 version that had worked forever for me without issue.. The system booted, and I figured I would try again. Again, I selected to update, the progress bar popped up... and about 5 min later, I got an email telling my my pool was in a degraded state. Afraid of leaving the window this time, I decided to wait until morning to see what had gone wrong, and went to bed, letting it finish the update.
I look in the morning, and it appears as though the update completed successfully, except when I click on anything in the Storage section on the left, nothing happens.. If I right click and open in new tab, it works fine, and will continue to work fine in that new window.
Now here is where things get interesting.. I open pool status, and it shows my pool status as DEGRADED, RaidZ2 Status as DEGRADED, and where it would normally say ada7p2, it now says "/dev/gptid/0e33a067-6b9e-11e4-a9c1-40167e65b1e6", and the status is UNAVAIL.
When I try to bring it online, it loads for a moment, and then stays on UNAVAIL.
When I try to bring it offline, it changes to OFFLINE.
In my search, I had found others had to resilver the drive when they had a hard drive fail, and it shows UNAVAIL, or when they accidentally put a boot partition on their drive, so I thought, okay, I wonder what will happen if I try to resilver the drive? But when I click on replace in the drop down, click the force check box, there are no options in the drop down box. it just says Member Disk*, and the REPLACE DISK button is greyed out.
I have also tried reseating the cables on the drive, and then even swapped the drive with another in the pool to see if it was a bad cable or something. No changes at all to the above.
I realize this could be a failed drive, but the timing just seems to coincidental that it would happen right after a failed update. At least to me it does.... Its also worth noting that I can see the drive in the bios, and it looks fine as well.
On top of all this, every once in a while now, I am getting booted from the web gui. It looks like the link status goes to down, and then comes back up on its own. This too only occurred after the failed update, but is kind of the least of my worries right now.
My hardware:
Motherboard: Asus P8B75-M
CPU: i3-2100 3.10 Ghz
RAM: 32GB (4 x 8GB)
Hard Drives: 8 x 4TB WD Black - Model: WD4003FZEX
Hard Disk Controller Card: Vantec 2 Channel Sata PCie Host Card UGT-ST622
Network Card: Just using the onboard nic
When I first boot up:
Code:
Apr 18 15:58:59 freenas kbd0 at atkbd0 Apr 18 15:58:59 freenas atkbd0: [GIANT-LOCKED] Apr 18 15:58:59 freenas coretemp0: <CPU On-Die Thermal Sensors> on cpu0 Apr 18 15:58:59 freenas est0: <Enhanced SpeedStep Frequency Control> on cpu0 Apr 18 15:58:59 freenas coretemp1: <CPU On-Die Thermal Sensors> on cpu1 Apr 18 15:58:59 freenas est1: <Enhanced SpeedStep Frequency Control> on cpu1 Apr 18 15:58:59 freenas coretemp2: <CPU On-Die Thermal Sensors> on cpu2 Apr 18 15:58:59 freenas est2: <Enhanced SpeedStep Frequency Control> on cpu2 Apr 18 15:58:59 freenas coretemp3: <CPU On-Die Thermal Sensors> on cpu3 Apr 18 15:58:59 freenas est3: <Enhanced SpeedStep Frequency Control> on cpu3 Apr 18 15:58:59 freenas ZFS filesystem version: 5 Apr 18 15:58:59 freenas ZFS storage pool version: features support (5000) Apr 18 15:58:59 freenas Timecounters tick every 1.000 msec Apr 18 15:58:59 freenas freenas_sysctl: adding account. Apr 18 15:58:59 freenas freenas_sysctl: adding directoryservice. Apr 18 15:58:59 freenas freenas_sysctl: adding middlewared. Apr 18 15:58:59 freenas freenas_sysctl: adding network. Apr 18 15:58:59 freenas freenas_sysctl: adding services. Apr 18 15:58:59 freenas ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled Apr 18 15:58:59 freenas ugen2.1: <Intel EHCI root HUB> at usbus2 Apr 18 15:58:59 freenas ugen0.1: <0x8086 XHCI root HUB> at usbus0 Apr 18 15:58:59 freenas ugen1.1: <Intel EHCI root HUB> at usbus1 Apr 18 15:58:59 freenas uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2 Apr 18 15:58:59 freenas uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 Apr 18 15:58:59 freenas uhub2: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 Apr 18 15:58:59 freenas uhub2: 8 ports with 8 removable, self powered Apr 18 15:58:59 freenas uhub0: 2 ports with 2 removable, self powered Apr 18 15:58:59 freenas uhub1: 2 ports with 2 removable, self powered Apr 18 15:58:59 freenas ugen1.2: <vendor 0x8087 product 0x0024> at usbus1 Apr 18 15:58:59 freenas uhub3 on uhub1 Apr 18 15:58:59 freenas uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus1 Apr 18 15:58:59 freenas ugen2.2: <vendor 0x8087 product 0x0024> at usbus2 Apr 18 15:58:59 freenas uhub4 on uhub0 Apr 18 15:58:59 freenas uhub4: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus2 Apr 18 15:58:59 freenas uhub4: 6 ports with 6 removable, self powered Apr 18 15:58:59 freenas uhub3: 6 ports with 6 removable, self powered Apr 18 15:58:59 freenas ugen0.2: <Lexar JumpDrive> at usbus0 Apr 18 15:58:59 freenas umass0 on uhub2 Apr 18 15:58:59 freenas umass0: <Lexar JumpDrive, class 0/0, rev 2.00/11.00, addr 1> on usbus0 Apr 18 15:58:59 freenas umass0: SCSI over Bulk-Only; quirks = 0x8100 Apr 18 15:58:59 freenas umass0:7:0: Attached to scbus7 Apr 18 15:58:59 freenas ugen0.3: <vendor 0x0c45 USB Wired Keyboard> at usbus0 Apr 18 15:58:59 freenas ukbd0 on uhub2 Apr 18 15:58:59 freenas ukbd0: <vendor 0x0c45 USB Wired Keyboard, class 0/0, rev 2.00/1.30, addr 2> on usbus0 Apr 18 15:58:59 freenas kbd2 at ukbd0 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 00 40 00 00 00 00 08 00 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): CAM status: ATA Status Error Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): RES: 51 04 39 00 00 00 00 00 00 02 00 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): Retrying command Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 00 40 00 00 00 00 08 00 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): CAM status: ATA Status Error Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): RES: 51 04 39 00 00 00 00 00 00 02 00 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): Error 5, Retries exhausted Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): SET_MULTI. ACB: c6 00 00 00 00 40 00 00 00 00 01 00 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): CAM status: ATA Status Error Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): RES: 51 04 00 00 00 00 00 00 00 01 00 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): Retrying command Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): SET_MULTI. ACB: c6 00 00 00 00 40 00 00 00 00 01 00 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): CAM status: ATA Status Error Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): RES: 51 04 00 00 00 00 00 00 00 01 00 Apr 18 15:58:59 freenas (aprobe0:ata2:0:1:0): Error 5, Retries exhausted Apr 18 15:58:59 freenas ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 Apr 18 15:58:59 freenas ada0: <WDC WD4000F9YZ-09N20L1 01.01A02> ATA8-ACS SATA 3.x device Apr 18 15:58:59 freenas ada0: Serial Number WD-WCC5DNHUSFV7 Apr 18 15:58:59 freenas ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) Apr 18 15:58:59 freenas ada0: Command Queueing enabled Apr 18 15:58:59 freenas ada0: 3815447MB (7814037168 512 byte sectors) Apr 18 15:58:59 freenas ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 Apr 18 15:58:59 freenas ada1: <WDC WD4000F9YZ-09N20L1 01.01A02> ATA8-ACS SATA 3.x device Apr 18 15:58:59 freenas ada1: Serial Number WD-WCC5DNHUSP9F Apr 18 15:58:59 freenas ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) Apr 18 15:58:59 freenas ada1: Command Queueing enabled Apr 18 15:58:59 freenas ada1: 3815447MB (7814037168 512 byte sectors) Apr 18 15:58:59 freenas ada2 at ata2 bus 0 scbus2 target 0 lun 0 Apr 18 15:58:59 freenas ada2: <WDC WD4003FZEX-00Z4SA0 01.01A01> ACS-2 ATA SATA 3.x device Apr 18 15:58:59 freenas ada2: Serial Number WD-WCC5D0009393 Apr 18 15:58:59 freenas ada2: 600.000MB/s transfers (SATA 3.x, UDMA5, PIO 8192bytes) Apr 18 15:58:59 freenas ada2: 3815447MB (7814037168 512 byte sectors) Apr 18 15:58:59 freenas ada2: quirks=0x1<4K> Apr 18 15:58:59 freenas ada3 at ata3 bus 0 scbus3 target 0 lun 0 Apr 18 15:58:59 freenas ada3: <WDC WD4003FZEX-00Z4SA0 01.01A01> ACS-2 ATA SATA 3.x device Apr 18 15:58:59 freenas ada3: Serial Number WD-WCC130502802 Apr 18 15:58:59 freenas ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) Apr 18 15:58:59 freenas ada3: 3815447MB (7814037168 512 byte sectors) Apr 18 15:58:59 freenas ada3: quirks=0x1<4K> Apr 18 15:58:59 freenas ada4 at ata3 bus 0 scbus3 target 1 lun 0 Apr 18 15:58:59 freenas ada4: <WDC WD4003FZEX-00Z4SA0 01.01A01> ACS-2 ATA SATA 3.x device Apr 18 15:58:59 freenas ada4: Serial Number WD-WCC5DEFNTCX8 Apr 18 15:58:59 freenas ada4: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) Apr 18 15:58:59 freenas ada4: 3815447MB (7814037168 512 byte sectors) Apr 18 15:58:59 freenas ada4: quirks=0x1<4K> Apr 18 15:58:59 freenas ada5 at ata4 bus 0 scbus4 target 0 lun 0 Apr 18 15:58:59 freenas ada5: <WDC WD4003FZEX-00Z4SA0 01.01A01> ACS-2 ATA SATA 3.x device Apr 18 15:58:59 freenas ada5: Serial Number WD-WCC5D6P4EXD1 Apr 18 15:58:59 freenas ada5: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) Apr 18 15:58:59 freenas ada5: 3815447MB (7814037168 512 byte sectors) Apr 18 15:58:59 freenas ada5: quirks=0x1<4K> Apr 18 15:58:59 freenas ada6 at ata5 bus 0 scbus5 target 0 lun 0 Apr 18 15:58:59 freenas ada6: <WDC WD4000F9YZ-09N20L1 01.01A02> ATA8-ACS SATA 3.x device Apr 18 15:58:59 freenas ada6: Serial Number WD-WCC5DRPX37R2 Apr 18 15:58:59 freenas ada6: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) Apr 18 15:58:59 freenas ada6: 3815447MB (7814037168 512 byte sectors) Apr 18 15:58:59 freenas random: unblocking device. Apr 18 15:58:59 freenas da0 at umass-sim0 bus 0 scbus7 target 0 lun 0 Apr 18 15:58:59 freenas da0: <Lexar JumpDrive 1100> Removable Direct Access SCSI device Apr 18 15:58:59 freenas da0: Serial Number AAIJMX89CF6VC5UX Apr 18 15:58:59 freenas da0: 40.000MB/s transfers Apr 18 15:58:59 freenas da0: 7648MB (15663104 512 byte sectors) Apr 18 15:58:59 freenas da0: quirks=0x2<NO_6_BYTE> Apr 18 15:58:59 freenas Trying to mount root from zfs:freenas-boot/ROOT/11.2-U8-1 []... Apr 18 15:58:59 freenas kernel: re0: link state changed to UP Apr 18 15:58:59 freenas kernel: re0: link state changed to UP Apr 18 15:58:59 freenas vmx_init: VMX operation disabled by BIOS Apr 18 15:58:59 freenas module_register_init: MOD_LOAD (vmm, 0xffffffff82a24470, 0) error 6 Apr 18 15:58:59 freenas GEOM_MIRROR: Device mirror/swap0 launched (2/2). Apr 18 15:58:59 freenas GEOM_MIRROR: Device mirror/swap1 launched (2/2). Apr 18 15:58:59 freenas GEOM_MIRROR: Device mirror/swap2 launched (2/2). Apr 18 15:58:59 freenas GEOM_ELI: Device mirror/swap0.eli created. Apr 18 15:58:59 freenas GEOM_ELI: Encryption: AES-XTS 128 Apr 18 15:58:59 freenas GEOM_ELI: Crypto: software Apr 18 15:58:59 freenas GEOM_ELI: Device mirror/swap1.eli created. Apr 18 15:58:59 freenas GEOM_ELI: Encryption: AES-XTS 128 Apr 18 15:58:59 freenas GEOM_ELI: Crypto: software Apr 18 15:58:59 freenas GEOM_ELI: Device mirror/swap2.eli created. Apr 18 15:58:59 freenas GEOM_ELI: Encryption: AES-XTS 128 Apr 18 15:58:59 freenas GEOM_ELI: Crypto: software Apr 18 15:58:59 freenas hwpmc: SOFT/16/64/0x67<INT,USR,SYS,REA,WRI> TSC/1/64/0x20<REA> IAP/4/48/0x3ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA,PRC> IAF/3/48/0x67<INT,USR,SYS,REA,WRI> UCP/8/48/0x3f8<EDG,THR,REA,WRI,INV,QUA,PRC> UCF/1/48/0x60<REA,WRI> Apr 18 15:58:59 freenas kernel: re0: link state changed to DOWN Apr 18 15:58:59 freenas kernel: re0: link state changed to DOWN Apr 18 15:58:59 freenas kernel: re0: link state changed to UP Apr 18 15:58:59 freenas kernel: re0: link state changed to UP Apr 18 15:58:59 freenas uhid0 on uhub2 Apr 18 15:58:59 freenas uhid0: <vendor 0x0c45 USB Wired Keyboard, class 0/0, rev 2.00/1.30, addr 2> on usbus0 Apr 18 15:59:03 freenas ntpd[4371]: ntpd 4.2.8p11-a (1): Starting Apr 18 16:00:55 freenas bridge0: Ethernet address: 02:7c:e4:5f:9d:00 Apr 18 16:00:55 freenas kernel: bridge0: link state changed to UP Apr 18 16:00:55 freenas kernel: bridge0: link state changed to UP Apr 18 16:00:55 freenas kernel: re0: promiscuous mode enabled Apr 18 16:00:55 freenas epair0a: Ethernet address: 02:46:d0:00:04:0a Apr 18 16:00:55 freenas epair0b: Ethernet address: 02:46:d0:00:05:0b Apr 18 16:00:55 freenas kernel: epair0a: link state changed to UP Apr 18 16:00:55 freenas kernel: epair0a: link state changed to UP Apr 18 16:00:55 freenas kernel: epair0b: link state changed to UP Apr 18 16:00:55 freenas kernel: epair0b: link state changed to UP Apr 18 16:00:55 freenas kernel: re0: link state changed to DOWN Apr 18 16:00:55 freenas kernel: re0: link state changed to DOWN Apr 18 16:00:55 freenas kernel: epair0a: promiscuous mode enabled
When I click on OFFLINE:
Code:
Apr 18 17:00:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=11215029481030843184 Apr 18 17:00:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=3976322218325219815 Apr 18 17:00:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=9630166916313623910 Apr 18 17:00:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=13675905590219508008 Apr 18 17:00:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=10433385203701266182 Apr 18 17:00:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=2070943494278390328 Apr 18 17:00:47 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=59876191846374754
When I click on ONLINE (and it changes to UNAVAIL):
Code:
Apr 18 17:01:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=11215029481030843184 Apr 18 17:01:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=3976322218325219815 Apr 18 17:01:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=9630166916313623910 Apr 18 17:01:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=13675905590219508008 Apr 18 17:01:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=10433385203701266182 Apr 18 17:01:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=14623635578535125442 Apr 18 17:01:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=2070943494278390328 Apr 18 17:01:46 freenas ZFS: vdev state changed, pool_guid=8688584271098232112 vdev_guid=59876191846374754
Sorry for the long post, I tried to be as detailed as I could and not leave out anything I thought would be helpful.
I have done my best to look for answers myself, I'm just at a loss on what could be causing this at this point. I am hoping it is something incredibly simple that I have just overlooked *fingers crossed*. Hoping someone can point me in the right direction!
Lastly, I apologize if I have posted this in the wrong area.. I know I have a few issues here, but the the degraded pool seems to be the worst of it.
Thank you!!
