Sidegrade causing endless reboots

[drdoug]

Help! I've had my FreeNAS system up and running for about a month now. I am running whatever the latest stable release. My hardware consists of a SuperMicro X9SCM-F, an i3-2100T, 8GB of ECC Kingston RAM (4x2GB sticks), 7 Western Digital RED drives, an ASMedia PCI-E SATA3 controller for the 7th drive.

Everything has been running well with no issues at all. I had just learned as I planned this that my system was not ideal as Intel retracted their option list on the i3-2100 to NOT include support for ECC. I have been running this same hardware for nearly 2 years with the exception of drives (4 new, 6 others removed), and controller - was running a Marvell-based SuperMicro SAS controller that FreeNAS didn't support. I have been running Windows Home Server 2011 with Drive Pool Extender.

Anyway, I ordered a Pentium G2030 which does have support for ECC and installed it. The board recognized the processor properly, but it didn't show the ASMedia BIOS during POST. After booting, I also confirmed that the drive attached to it was not available. I reseated the processor with no changes. I updated the BIOS on the board to the latest version, which is 2 months old, and it had no change. I let the system boot and before I could do much I lost the web interface. I found that it had rebooted. It booted again, but rebooted before I could shut it down. I swapped back to the i3 processor I had in there previously and it saw the ASMedia controller. The interesting thing about that is that the ASMedia controller is on one of the x4 PCI-E 2.0 slots from the south bridge, not one of the PCI-E slots from the processor itself. I'm not sure why it would not recognize the card, but otherwise POST without errors and seem to function okay except for the reboots.

I started a resilver on the drive that had disappeared while the Pentium was in place. Before I could connect to the console to get a status on the process, it rebooted again. Basically, it will not remain booted longer than about 10 minutes. I watched it closely at the console itself one time and it rebooted the moment the console options appears post boot. It never fails to boot successfully, and I see no error messages during boot, it just will not stay running.

I am currently running MemTest86, but it has not found any errors in it's first pass. I tried selecting a prior release boot option and then selected the option to update. After doing so, it still operated in the same manner. I'm not sure if it rewrote the latest version, or if it failed while applying the update and then just booted to the default latest version.

I know just enough to get me in trouble. I know I could try starting from scratch and importing my pool and setting everything back up. I'm not sure how it would handle the resilver that is in progress, nor do I know if it would resolve things. I'd sure like to find a way to fix this in place.

Any thoughts?

 

[SweetAndLow]

Any log messages? Could also be a bad power supply or memory. Does your bios have any events logged?

 

[drdoug]

I can't seem to get in long enough to look at logs. If I can get connected and grab them quickly, which specific logs would I want to be checking?

The BIOS does not show any error logs, and I made sure that logging was enabled.

The case is also a SuperMicro with 900W redundant power supplies. I only run one because they are extremely inefficient using 35W just to standby. I will try switching to the other one and see if that makes any difference. I believe the supplies themselves are just 12V, with the backplane providing the 3.3V, 5V, and -12V. I know it's not very accurate, but IPMI shows voltages that are pretty spot on.

I'll reply back if swapping them changes anything.

Thanks

 

[marbus90]

And removing the card altogether solves everything? (I don't trust anything but LSI SAS 2008/2308/3008 or Intel AHCI SATA) There was some weirdness with Socket 1155 about specific PCIe slots only available with Xeon processors with more PCIe lanes.

 

[drdoug]

No, during the whole process the card has remained in the same spot. It does beg a question that I do not know the answer to. All socket 1155 processors (xeons included) only have 16 PCI-E lanes for expansion. There are another 4 linked to the PCH, and the PHC itself can provide up to additional 4 for peripherals. The block diagram for the X9SCM shows 4 expansion slots. All x8. The first two are electrically x8 slots, which should consume all 16 lanes the processor can provide. However, the diagram in the manual shows the third slot (labeled slot 5) can provide an electrical x4 connection. The final slot provides the x4 from the PHC. My card, due to clearance issues, is in slot5, or the one that seems the odd man out. When the i3 is installed, the card is detected. When the Pentium is installed, it is not.

Incidentally, I put in a different USB stick and reloaded then imported the pool. This seems to have resolved the issue as it completed resilvering and the system has been running for 11 hours now.

I would love to be running an LSI controller, but purchasing the drives necessary to expand my system drained my funds. I will eventually replace it, but it has been running without issue for the past month, and for a year prior in a different system.

 

[marbus90]

http://ark.intel.com/compare/65693,52269,75052,65734

look at PCIe configuration. in theory that slot5 shouldn't be working.

 

[drdoug]

I know, right? I've been previously using that slot for an x4 connection for my SuperMicro controller used before FreeNAS. I can make some available space and move the controller, but I don't understand why it would work with the i3.

 

[drdoug]

I've had more time to play with my configuration. I put the Pentium G2030 in the system, reset the BIOS and tried again. The BIOS for my secondary controller once again did not show up. I proceeded to the installer to verify that the disk wasn't detected, and confirmed that it wasn't. I tried the add-in card in every slot. I was quite sure that it would at least work in the PCI 2.0 slots provided by the south bridge, but no. It was not detected in any slot.

I added back my SuperMicro SASMV8 card just to see if it was detected. Surprisingly, it worked fine in every slot - even slot 5 which I still don't understand why it ever worked there.

There is obviously an issue with the ASMedia card and the Pentium processor with the current BIOS release I am on. I even tried forcing PCI-E revisions from 3.0, 2.0, and 1.0. Nothing changed. I know it would be ideal for me to be running the LSI card, but it's just not in the budget at the moment. Unfortunately, the card I could get to work is one I trust even less than the ASMedia - a Silicon Image 3132 flashed to non-RAID mode.

Am I shooting myself in the foot here? I know that this silicon has been out there a long time, and it's firmware should be pretty mature by now. I just have never been impressed with Silicon Image adapters.

Why doesn't Intel create their own add-in cards? LSI may be great, but like in NICS, their SATA powered ports seem to be the gold standard.

I wish I would have created my array with just 6 disks in a RAIDZ2 config instead of 7. I have 5 TB of free space, so I could have gotten by and my space usage isn't increasing at a dramatic rate. Unfortunately, rebuilding isn't an option as the drives that my data used to be on, and that I could use to temporarily hold it while I recreated it have been sold. I have one single 2 TB drive that isn't being used, but 10 GB of data. I backup all critical data (Pictures, documents, anything original) to 2 local external drives that are only connected during backed, and a second backup offsite to my dad's house. Unfortunately, even though I could survive without the rest of the data, it's ~400 movies which I really do not want to reconvert.

 

[drdoug]

One thing I got cleared up is how slot 5 works. The C204 chipset is different from the desktop equivalent, in that is supplies 8, instead of 4 PCI-E 2.0 lines for expansion. Slot 4 and Slot 5 are driven by the PCH electrically as x4 slots with x8 connectors. Slot 6 and 7 are clearly understood as each being 8 lines from the 16 coming directly off the processor. It is not dependent on it being a Xeon, an i3, a Pentium, or even a Celeron. All the expansion slots are available. http://ark.intel.com/products/52804