FreeNAS will not boot after adding hard drives

[strudinox]

Hello!

I've got a stable FreeNAS setup working perfectly fine using 8x 2TB drives on Z2. I wanted to create a second zpool using 4x 6TB drives in Z1, however when I add the drives in the system, FreeNAS will not boot. I'm currently booting from the USB drive (mirrored across 2 drives). All four of the 6TB drives are recognized in BIOS and my boot order is unchanged, booting from USB. When the drives are added, I get the generic error: "This is a FreeNAS data disk and cannot boot system".

Hardware:

• Supermicro H8SGL-F Motherboard
• 4x 8GB DDR3 ECC Reg
• AMD Opteron 6234 12-Core
• 2x IO Crest 4 Port SATA III PCIE controller (for the 8x 2TB drives)

Any help would be greatly appreciated!

 

[nightshade00013]

It's a BIOS problem, when you added the disks and put something on them the bios said "Hey I can now read something" and then tries to boot it.

Change the boot order and the problem will be solved.

 

[strudinox]

It's a BIOS problem, when you added the disks and put something on them the bios said "Hey I can now read something" and then tries to boot it.

Change the boot order and the problem will be solved.

That was my first thought, and I triple checked that USB device was first. I even tried disabling HDD boot devices and no luck. :(

 

[nightshade00013]

Turn off "Boot other devices" since you have a addon card of dubious quality it is probably asserting itself first in the lineup.

BTW you realise it's using a marvell controller and not recommended right?

 

[strudinox]

Turn off "Boot other devices" since you have a addon card of dubious quality it is probably asserting itself first in the lineup.

BTW you realise it's using a marvell controller and not recommended right?

I'll give that a try. I have heard that Marvell isn't recommended but I haven't had any issues with so far. The four drives I'm trying to add are connected directly to the SATA ports on the motherboard.

 

[strudinox]

Update: Now this is weird... If I unplug one of the drives, FreeNAS boots just fine! Power off, plug it back in, doesn't boot. Doesn't matter which drive I unplug as long as there are only 3, it works. Scratching my head on this one...

 

[nightshade00013]

The I suspect there is an issue in the BIOS, May want to check and see if there is an upgrade or contact SuperMicro directly. This being an AMD based board and controller it will be less than likely that many here will have any information on the subject. The Onboard ports do support some raid functions so you may attempt to change the drives into an IDE mode but at the very least make sure they are in AHCI mode if they are not already.

However I saw where someone dealing with some ESXI hosts were having issues with the same board. You may end up needing to just get some SAS controllers and discontinue using the onboard ports.

 

[farmerpling2]

I have a test system running AMD and this will happen when adding new disks. You have to reorder the boot order or disable drives that can be booted to. I just make sure to disable all drives but the one I want to boot from.

 

[strudinox]

Agreed, it's starting to sound like a BIOS issue. I checked and the BIOS is up to date. I tried different modes (IDE, AHCI, RAID) and none of the modes fixed the issue. I'll probably just order another controller if I can't get this working.

 

[farmerpling2]

Look at the boot section of BIOS. I would be surprised if it is not. Make sure you disable all devices BUT your device you want to boot. Make sure you only have the one device in the boot list. When you add new devices, this tends to cause the list to get reset...

 

[strudinox]

Look at the boot section of BIOS. I would be surprised if it is not. Make sure you disable all devices BUT your device you want to boot. Make sure you only have the one device in the boot list. When you add new devices, this tends to cause the list to get reset...

Thanks, I'll give that a try tomorrow and report back. I did however have all but USB and CDROM enabled so not sure if that was enough or not.

 

[wblock]

Turn off "Boot other devices" since you have a addon card of dubious quality it is probably asserting itself first in the lineup.

This. On some cards, the booting ability can be disabled through a menu. On my LSI cards, I just didn't install the boot code (option ROM or BIOS or whatever they call it). The card runs fine, but the system does not attempt to boot from any of the drives attached to it.

 

[strudinox]

UPDATE: I purchased another PCIE SAS card to plug the 4x 6TB drives into rather than using the motherboard. The result: Same issue! When all four of the drives are installed in the system, I get a boot device error, press any key to blah blah.... Detach one, and FreeNAS boots. I went into BIOS and made 100% sure that ONLY the USB bootable device was active. Unfortunately I don't have an option for "Boot other devices" but all other devices are disabled according to BIOS.

The drives work perfectly fine all together outside the system in another PC. I did make sure all drives were wiped clean and don't contain any data that might be messing with FreeNAS.

At a loss with this system at this point...


EDIT: I also tried plugging 2 drives into the PCIE controller, and 2 into the motherboard, same issue. Also tried hitting F3 at POST to manually select the boot device. My SanDisk Cruiser came up, and still was not recognized as bootable with all drives installed.

 

[farmerpling2]

Some thoughts in no particular order...

1. Are the USB drives on the same controller or a different one? One 2.0 other 3.0?
   
2. Physically remove one USB drive to continue diagnosing the problem. Does problem go away? Move USB drive to different USB ports, does anything change?
   I have seen a problem like this when I had controllers on same USB controller. I moved one to 2.0 and other to 3.0 and it worked.. (I think that is what I did - Might have it backwards.)
   This is a USB drive PORT problem, more than likely... I had this one one AMD system running FreeNAS, now that I think about it.
   
3. Try moving the USB drive to all of the USB ports, correctly set the boot device in BIOS and can you boot up on the USB drive.
4. Put a bootable hard drive into the mix and see what port is actually trying to be booted from. From onboard controller or PCI controller?
5. Have you disabled the boot ROM on the PCI SAS controller?
6. Turn off network booting, just in case (probably will do nothing but trying to minimize).

Let us know what you find.

 

[strudinox]

Thanks for your suggestions farmerpling. I attempted to shuffle around the USB drives, currently they were on two different controllers. Tried putting them on the same controller, and booting up with just a single dive. This didn't seem to have any affect on the main issue. Boot ROM is currently disabled as well. It's so strange because the BIOS sees the USB drive, and even gives me the option to boot from USB when I press F3 to manually select the boot device. Once it tries, I get the error that no boot device was found. I'm starting to think this has to be a bug in the BIOS.

 

[Inxsible]

I have had a similar issue with my Tyan S5533 and my Supermicro X9SCL-F boards. After you connect the drive that you have a problem with, go into the BIOS and set up USB to be the first boot device. Then in the Exit menu -- save it as User Defaults. For some reason, after saving it as User Defaults, the preference to boot from USB sticks. Otherwise it would revert back to the SATA0 drive and keep telling me that this is a FreeNAS data drive and system cannot boot.

 

[Inxsible]

I wonder why the forum added a link to my Supermicro board. I sure didn't. Also the link takes you to an amazon page that doesn't exist. Stop doing that forum !!

 

[nightshade00013]

I wonder why the forum added a link to my Supermicro board. I sure didn't. Also the link takes you to an amazon page that doesn't exist. Stop doing that forum !!

It's forum monetization, Can't blame iX for trying to cap a few bucks. At least it's not huge banner ads all over the place.

 

[danzaslap]

Thanks for your suggestions farmerpling. I attempted to shuffle around the USB drives, currently they were on two different controllers. Tried putting them on the same controller, and booting up with just a single dive. This didn't seem to have any affect on the main issue. Boot ROM is currently disabled as well. It's so strange because the BIOS sees the USB drive, and even gives me the option to boot from USB when I press F3 to manually select the boot device. Once it tries, I get the error that no boot device was found. I'm starting to think this has to be a bug in the BIOS.

Hi Did you ever solve this issue - i'm in the same boat :(

 

[strudinox]

Hi Did you ever solve this issue - i'm in the same boat :(

I unfortunately never was able to resolve the issue. I abandoned this track and am slowly replacing the 2TB drives with 6TB in which I'll eventually expand the pool for more storage. If you ever figure it out, I would definitely be interested in how you fixed it considering all the trouble it gave me!