Unable to mount root after clean install

[mjanssen]

Hi!

I'm quite new to FreeNAS, so bear with me when I ask silly questions. I know a little Linux, but nothing fancy

I did a clean install of FreeNAS 11, installation went well, until the first reboot. When using "Normal Boot" the root fails to mount. When manually selecting "Verbose", all mounts well and everything works.

Strange thing is that it does not always fail in "Normal Boot", but it does work every time in "Verbose".

Looked at several options, and found multiple posts on the forum, but I haven't been able to find the solution.

Tried to change GRUB to use Verbose by default, but wasn't able to find the right files and if I would have been, I wouldn't know what to change. ;-)

** UPDATE ** I forgot to mention FreeNAS is installed on a RAID5 HDD volume. Not SD-Card or USB.

Hopefully someone can point me in the right direction.

If you need more info, please feel free to ask.

Regards,

Martijn

 

[dlavigne]

To assist, we'll need details about your hardware as per the forum rules.

Also, don't install on your storage disks, you do want the OS to reside on a separate boot device.

 

[mjanssen]

Hi,

I understand the rules, but the fact is that I don't exactly know what kind of hardware it has. It is an old SAN that ran Open-E. It contains an Adaptec 51245 card with 12 x 3Tb Hitatchi HDD's.

Is there a command/tool I can run to show the hardware in FreeNAS ?

FreeNAS is installed on a separate boot volume, but it is HDD.

The intriguing fact is why it does boot and functions normally in verbose mode and not in normal mode.

 

[dlavigne]

Is there a command/tool I can run to show the hardware in FreeNAS ?

freenas-debug -h (see http://doc.freenas.org/11/cli.html#freenas-debug).

 

[mjanssen]

Great, thanks!

Output attached in debug.txt.

 

[bw1faeh0]

Do you have any other devices (USB-Stick, etc) with an (old) FreeNAS installation that are connected to the PC while booting?

In my case FreeNAS fails to mount the root file system if an other installation is connected to the PC (old USB-Stick, small internal SSD with a second FreeNAS). I have to remove all other installations to successfully mount the root file system and continue the boot process.

 

[wblock]

FreeNAS is installed on a RAID5 HDD volume. Not SD-Card or USB.

This is unclear. For FreeNAS to work as it was designed, the drives should be directly accessible (aka "JBOD"). A hardware RAID card prevents that.

 

[mjanssen]

Do you have any other devices (USB-Stick, etc) with an (old) FreeNAS installation that are connected to the PC while booting?

In my case FreeNAS fails to mount the root file system if an other installation is connected to the PC (old USB-Stick, small internal SSD with a second FreeNAS). I have to remove all other installations to successfully mount the root file system and continue the boot process.

I found this issue in a thread on the forum, checked that and is not the issue unfortunately.

 

[mjanssen]

This is unclear. For FreeNAS to work as it was designed, the drives should be directly accessible (aka "JBOD"). A hardware RAID card prevents that.

FreeNAS sees the hardware RAID volumes without any issue. But sees no individual physical drives. All volumes are shareable and accessible.

And FreeNAS functions flawlessly, with all storage, but only when started in Verbose mode.

 

[ReadyNAS2018]

FreeNAS sees the hardware RAID volumes without any issue. But sees no individual physical drives. All volumes are shareable and accessible.

And FreeNAS functions flawlessly, with all storage, but only when started in Verbose mode.

I know I'm new to this forum, but have been using FreeNAS for a while and I thought that FreeNAS should not be used with a hardware RAID. My understanding (someone correct me if I'm wrong) is that as it's hardware RAID then it would see the RAID as one disk. This is why you can't use one of the drives in the RAID array as your boot disk for FreeNAS.

 

[Nick2253]

Can you provide the output of

zpool status -v

 

[Nick2253]

I know I'm new to this forum, but have been using FreeNAS for a while and I thought that FreeNAS should not be used with a hardware RAID. My understanding (someone correct me if I'm wrong) is that as it's hardware RAID then it would see the RAID as one disk. This is why you can't use one of the drives in the RAID array as your boot disk for FreeNAS.

This is correct; FreeNAS (or more specifically, ZFS) should not be used with hardware RAID. The biggest reason is because the self-healing scrubbing and copy-on-write protections of ZFS are completely lost when using hardware RAID: ZFS does not believe there is any redundancy on the disk, so when data goes bad, it does not know that it can be recovered, and the RAID card reintroduces the write hole when it puts data on the disk. There are also a variety of performance issues that can crop up with hardware RAID cards. If you must use hardware RAID, then I would strongly suggest using a different filesystem than ZFS, which means using a different system than FreeNAS.

I believe that filesystems like ZFS are the future of data storage: as magnetic disks get larger and larger, it's becoming more important to have highly redundant self-healing file systems. And the added cost (and added problems, like CoW) of dedicated RAID controllers are largely a thing of the past thanks to the significantly increased speeds of CPUs, RAM, and HDDs, which means that software RAID is just as fast, and significantly more capable.

 

[mjanssen]

Can you provide the output of

zpool status -v

I can post that on monday, I don't have access to the system until then.

 

[mjanssen]

This is correct; FreeNAS (or more specifically, ZFS) should not be used with hardware RAID. The biggest reason is because the self-healing scrubbing and copy-on-write protections of ZFS are completely lost when using hardware RAID: ZFS does not believe there is any redundancy on the disk, so when data goes bad, it does not know that it can be recovered, and the RAID card reintroduces the write hole when it puts data on the disk. There are also a variety of perfomance issues that can crop up with hardware RAID cards. If you must use hardware RAID, then I would strongly suggest using a different filesystem than ZFS, which means using a different system than FreeNAS.

I believe that filesystems like ZFS are the future of data storage: as magnetic disks get larger and larger, it's becoming more important to have highly redundant self-healing file systems. And the added cost (and added problems, like CoW) of dedicated RAID controllers are largely a thing of the past thanks to the significantly increased speeds of CPUs, RAM, and HDDs, which means that software RAID is just as fast, and significantly more capable.

The RAID controller can also create a JBOD. If you think that will solve my issue, I'll happy to give it a try.

 

[Nick2253]

The RAID controller can also create a JBOD. If you think that will solve my issue, I'll happy to give it a try.

I don't think this will solve your issue, but I strongly recommend doing this because it will significantly improve your setup.

Make sure that JBOD is actually JBOD, and not just each drive individually passed through as a RAID0 volume.

 

[mjanssen]

I don't think this will solve your issue, but I strongly recommend doing this because it will significantly improve your setup.

Make sure that JBOD is actually JBOD, and not just each drive individually passed through as a RAID0 volume.

Ok, will keep that in mind.

 

[wblock]

FreeNAS sees the hardware RAID volumes without any issue. But sees no individual physical drives. All volumes are shareable and accessible.

And FreeNAS functions flawlessly, with all storage, but only when started in Verbose mode.

The hardware RAID masks the drives from ZFS. ZFS will think there are no problems until it is too late. And there can also be problems with a hardware RAID controller putting metadata on a drive making the data on that drive invisible unless used with that model of hardware RAID.

So it might look to be working fine. But it has a way of failing badly at bad times.

I was thinking there might be something set with the hardware RAID that has staggered drive spin-up or possibly just slow enumeration, making it not ready when the boot is ready. A verbose boot takes longer, maybe just long enough so that the controller is ready by the time it finishes.

If you can post exactly what you see when it fails to boot, that would help. If it is a timing problem, just pressing Enter might be enough to continue the boot at a "mountroot" error.

 

[mjanssen]

I re-installed FreeNAS with a JBOD in the RAID controller and second installed it on a separate USB device.

All seems to run smoothly now, will do some more testing in the next couple of days.

Thanks for all your support!