VM broken, why, how to fix, how to prevent?

[Kuo]

Hi,

I am running with FreeNAS-11.1-U2 (btw, is it easy/risky/whatever to update ?).

2 VMs are running in 2 differents pools, both with Debian Stable.

FreeNAS has been shutdown twice brutally : the first time, somebody pressed the big red button the second time, there was a power failure.
These stops have (probably) caused (I don't know which one) the breakage of 1 VM.
Here is what I got now when I start the VM :

Could these cuts really have caused this?
How can I prevent this problem in the future?
Can I get my VM back? How?

Thank you! :D

 

[sretalla]

Type exit at the prompt to go to bios. Navigate the boot options to boot from the correct EFI.

There's a thread around somewhere which discusses how to update your install properly to avoid that. I never paid too much attention to it, so haven't saved the reference.

 

[Mlovelace]

Hi,

I am running with FreeNAS-11.1-U2 (btw, is it easy/risky/whatever to update ?).

2 VMs are running in 2 differents pools, both with Debian Stable.

FreeNAS has been shutdown twice brutally : the first time, somebody pressed the big red button the second time, there was a power failure.
These stops have (probably) caused (I don't know which one) the breakage of 1 VM.
Here is what I got now when I start the VM :

Could these cuts really have caused this?
How can I prevent this problem in the future?
Can I get my VM back? How?

Thank you! :D

This blog post explains on how to fix the UEFI file level to get your VM to boot properly.

https://www.jongibbins.com/solving-uefi-boot-problems-on-bhyve-freenas-vm/

 

[Kuo]

Thank you for your answers.
I can't get my VM back, it seems to be a deeper problem...
It doesn't matter, there wasn't (at this time) much things.

How can I prevent such an incident in the future? Clone ? Snapshot ? Both ? Other ?

Thank you :D

 

[jgreco]

Not being snarky here, just blunt - I assume you "know" this but it bears repeating:

1) Protect the big red button.

2) Use a UPS and have it communicate status with FreeNAS.

VM's are notorious for getting screwed up when a hypervisor or storage platform goes away unexpectedly. Basically many OS's will have I/O cached in memory and may not flush it out to disk appropriately.

Some more discussion of that is buried in

https://www.ixsystems.com/community/threads/some-insights-into-slog-zil-with-zfs-on-freenas.13633/

which is a good prelude to the next bit of advice.

3) Enable sync writes and add a SLOG if the performance hit is too dramatic for you. This will help reduce the chance of damage to your VM images.

4) Take periodic snapshots so that you may be able to recover from an older checkpoint. Note that snapshots of running VM's have their own caveats and are not guaranteed to be reliable or recoverable.

5) Use a completely different method, such as a file-based backup, ON the VM itself to back up the contents somewhere safe.

Loss of VM contents is every admin's nightmare. Fortunately there's a lot you can do to help proactively address the problem.

Good luck!

 

[Kuo]

Thank you for your answer, I'll try to apply these advices in a few days.
This bloody button was supposed to be disabled... I'll see why it wasn't. Or maybe unplug it.

Otherwise, I believe I am doing something wrong with my VM.

I made a new VM again.
Then, I updated to FreeNAS-11.1-U7. And go back to the first screenshot.
I thought maybe I should have turned off my VM before updating the system.
OK, let's try again, I don't surrender.
Installation finished, I stop the VM, delete CD-ROM and restart :

Here are the steps I follow to create my VM:

• create new zvol (approx. 20GiB)
• I create a VM by leaving everything by default
  • I change the network card to virtIO (because I was told it's better to do so)
  • I activate the VNC in 800*600
  • I create a disk, put my zvol in it and leave everything by default
  • I create a CDROM and put my ISO (debian 9.9 amd64 netinstall)
  • start install
  • once finished I turn off and delete the CD-ROM (otherwise it will reboot on it)
  • and see the screenshot above

What am I doing wrong? Or what could be the problem?

Thank you! :D

 

[KrisBee]

@Kuo

When you go through the VM wizard can either choose the option to allow the system to create the Zvol used as the VM's disk drive, or you select one you have ALREADY created on your pool. To avoid problems with the UEFI boot method, select the "Advanced Options" to install your VM. When your reach the point of installing grub make sure you answer yes the question "Force grub-efi installation to the removable media path".

Once you've installed the VM, and stopped it to the remove CD-ROM. you will probably loose network connectivity on the first re-boot. This happens as the virtual NIC slot has changed, Check the output of "ip a" for the correct NIC name and edit /etc/network/interfaces to match this.

 

[Kuo]

@Kuo
To avoid problems with the UEFI boot method, select the "Advanced Options" to install your VM. When your reach the point of installing grub make sure you answer yes the question "Force grub-efi installation to the removable media path".

OK, it works !
But I don't understand why. I mean, with Debian 9.5 and 9.8, I had no problem. Why now ?