Resilvering not completing and box keeps rebooting

[unholyrattrap]

I have lost a drive and run through the replace wizard after identifying the proper drive. With this rebuild its a brand new out of the box retail drive that I am adding the system. However the resliver has yet to complete, and the freenas box keeps rebooting when it gets to 10% resliver.

I will attach debug info here but am looking for a fix, or a suggestion on how to get the resliver to complete and the box to stop rebooting. Ideally the resliver over the reboot. Let me know if you need additional info to help but it is getting frustrating that I am on day 5 of rebuilding a single drive at a nice .05%.

Current System Snapshot info:
Build FreeNAS-11.0-U4 (54848d13b)
Platform Intel(R) Core(TM) i3-2100 CPU @ 3.10GHz
Memory 16033MB

 

[SweetAndLow]

Does your new drive even work? I should be tested for 3 days before using it to resilver.

This sounds like a hardware failures of some kind. Or possibly metadata corruption but usually people have terrible pools when that happens and yours looks fine.

Since you didn't provide all your hardware info and how your disks are attached no one can really help you anymore.

 

[unholyrattrap]

Hardware:

• Case: Athena Power RM-4U4243HE12 12Gb/s 4U Hot-Swap 24-Bay E-ATX Rackmount Server Chassis w/ 12 Gbps Mini-SAS Backplane
• Raid Card: Adaptec 78165
• Raid Card is connected to the sas backplane with 12gbps mini sas cables
• Motherboard: Asus P8P67
• Ram: Kingston Ballistix 16GB in two 8GB Sticks
• Each row takes a single minisas back to the Adaptec so that the Adaptec can access each drive independently
• The Adaptec is set to HBA pass-through mode so that its just used as a super expensive relay to the minisas backplanes. I was originally going to build the OS out differently an was persuaded to try FreeNas.

Do you need more than that? Otherwise, I'll just scrap this whole thing then once I can get the data off and write this off as a failed experiment as rebuilds, scrub issues, and premature drive failures have been more of a problem than it's been worth to be honest.

 

[nightshade00013]

Was nosing around and also found you had the following issue with the server earlier:

https://forums.freenas.org/index.ph...ant-clean-it-thoughts-on-how-to-fix-it.53109/

Was that ever fixed? It may stem from that issue and/or consumer grade parts without ECC ram.

 

[unholyrattrap]

It was fixed, I had some bad ram in that build that I pulled with good ram and those issues went way.

 

[SweetAndLow]

as rebuilds, scrub issues, and premature drive failures have been more of a problem than it's been worth to be honest.

These are not your problems. Your problems are from using poor quality hardware that is either old and failing like your hdd's or a raid card that is clearly not supported. I suspect your raid card might be causing timeouts and causing the resilver to restart because writes are failing. We should figure out what driver your card is using, if it's the mpr or mps then things are probably ok but if it' the mfi that is what is causing your problems.

Output of:
dmesg | grep mpr, or dmesg | grep mps, or grep dmesg | grep mpt
sas2flash --listall

 

[unholyrattrap]

dmesg | grep mpr kicked back nothing, and sas2flash
# dmesg | grep mpr
# sas2flash --listall.
LSI Corporation SAS2 Flash Utility
Version 16.00.00.00 (2013.03.01)
Copyright (c) 2008-2013 LSI Corporation. All rights reserved

No LSI SAS adapters found! Limited Command Set Available!
ERROR: Invalid command --listall.

Exiting Program.

 

[SweetAndLow]

dmesg | grep mpr kicked back nothing, and sas2flash
# dmesg | grep mpr
# sas2flash --listall.
LSI Corporation SAS2 Flash Utility
Version 16.00.00.00 (2013.03.01)
Copyright (c) 2008-2013 LSI Corporation. All rights reserved

No LSI SAS adapters found! Limited Command Set Available!
ERROR: Invalid command --listall.

Exiting Program.

This is exactly your problem. You can not use a raid card, HBA is the only safe way to have a zfs system. Get a HBA off ebay for $100 and you will have a 100% better experience. While you are there get a new motherboard that is actually supported and does not have a realtek nic. If you have not ran into networking problems yet you will soon.