SOLVED Replacing Boot Drives results in: This is a NAS data disk and can not boot system. System halted.

[Scampicfx]

Dear guys,

my goal was to replace two mirrored supermicro sata-dom-chips by two mirrored intel ssds (boot-vdev).

What did I do?

System -> Boot -> Status
I selected first sata-dom chip 16gb, I clicked replace and selected first new intel ssd 120 gb -> I clicked on "Replace Disk"
I selected second sata-dom chip 32gb, I clicked replace and selected the second new intel ssd 120 gb -> I clicked on "Replace Disk"

This is what my server looks like after replacement proccess:

Just for testing purpose, I selected "ada0p2", and clicked another time on "Replace Disk". As you can see, I only have a choice of disks which are not used right now:
nvd0 = intel pcie ssd
ada2 = supermicro sata dom 16 gb
ada3 = supermicro sata dom 32 gb

So, I thought the replacement proccess was successfull... But...

I kept this system running for a few weeks. Everything looked good so far. The "WebGUI reporting section" gave me the impression that only intel ssds are used - and no sata-dom chips!.

What I did next was powering down the system, removing the sata-dom chips, then powering on again.

I was a bit surprised when I saw following message on the boot screen a few moments later:

This is a NAS data disk and can not boot system. System halted.

What did I do in order to avoid this message?

I entered BIOS and noticed following boot order:
#1 hgst disk
#2 intel ssd 1
#3 intel ssd 2

Alright, I thought this is the mistake. So I changed it to:
#1 intel ssd 1
#2 intel ssd 2
#3 hgst disk

However, this did not solve this issue!

I installed sata-dom chips again and changed boot order to:
#1 sata-dom 1
#2 sata-dom 2
#3 hgst disk
#4 intel ssd 1
#5 intel ssd 2

... and the system is booting again! Right now I'm in FreeNAS and I just took the screenshot you can see at the beginning of my posting. Well, I have absolutely no idea, how this can happen? As you can see, FreeNAS marks both sata-dom chips as being "unused" and "available for disk replacement", but when they are not installed in my server, I'm not able to boot.

I was a bit scared when booting from the sata-dom-chips, because I thought that this will boot up my FreeNAS config from a few weeks ago, so to speak my freenas config BEFORE disk replacement. However this did not happen! Right now I'm looking at an up2date FreeNAS config with all recent changes I made. So, obviously, although I selected sata-dom-chips as primary boot device, I'm now booted up using Intel SSDs, am I?

So, I guess I made a big mistake somewhere. But I don't know where :( Right now, it looks to me that the Intel SSDs are used, however, FreeNAS requires the Satadom chips to make some sort of initial boot proccess. But, I can't explain this because the disk replacement should perform a 1:1 copy to new disk, shouldn't it?

So in case anyone of you has some advise or any tipps, I would really appreciate it! I would like to remove sata-dom chips and boot the server with intel ssds. The problem at this point is, that the intel ssds are already in use, as you can see at the screenshot at the beginning of my posting.

Thanks so much for your time reading this posting!

Cheers, Chris

 

[joeschmuck]

Instead trying to troubleshoot this issue as it would take time to get things right I'd rather offer you a simple solution since your system is presently running...

1) Backup your configuration file.
2) Power down your FreeNAS.
3) Remove your SATA DOM's.
4) Connect only a single SSD to SATA Port 0.
5) Install FreeNAS on the SSD.
6) Restore the config file.
7) Be Happy!

You do not need a mirrored SSD, these are very reliable. The config file is backed up automatically on the pool just in case you ever need it but it's a manual operation to obtain it, but it's there.

If you really want a mirror then install both and then install FreeNAS and tell it to mirror the drives.

All I can say is something is going on when you installed the SSDs and this should correct any issue you have.

 

[Scampicfx]

Hey,
thanks for your answer!

Just to be sure: May all other HDDs stay attached to the enclosure during reinstall of FreeNAS?

 

[danb35]

May all other HDDs stay attached to the enclosure during reinstall of FreeNAS?

Yes, just make sure you don't select any of them as the disk to install to at this screen:

 

[joeschmuck]

Hey,
thanks for your answer!

Just to be sure: May all other HDDs stay attached to the enclosure during reinstall of FreeNAS?

But if you are not sure of yourself you can physically disconnect them when you build the boot drive and then once FreeNAS boots up, reconnect the drives after step 5 but before step 6 indicated above. But when selecting the drive it should be obvious by the drive size alone.

 

[Scampicfx]

Hey, thanks. I just did a reinstall of FreeNAS. I monitored the proccess closely. It looks like everything is fine now!
Thanks for all your help! I really appreciate all your valuable input in this forum! :)

 

[joeschmuck]

Hey, thanks. I just did a reinstall of FreeNAS. I monitored the proccess closely. It looks like everything is fine now!
Thanks for all your help! I really appreciate all your valuable input in this forum! :)

Glad it worked out for you.