bhyve VM exited on signal 10

[jclendineng]

Specs? Thats probably a memory issue....

 

[ryanakata]

Specs? Thats probably a memory issue....

C2750D4i
16gb ECC RAM
Custom printed fan shroud with a noctua fan to keep it cool.
3x 8TB WD Reds
3x 5TB WD Red
1x 250GB SSD for Jails/VM storage
2x Sandisk Flash storage for OS
Fractal Node 304 w Corsair SFX PSU.

Memory issue could be logical but does seem strange it worked perfectly on 11.1 and immediately stops on 11.2.

Edit to include VM specs. Tried with 1 and 2 vCPUs with 1024-2048mbMB of ram and the issue persists through all configurations.

 

[nixfiction]

I am seeing this same issue on 11.2. I have 3 x VM's that migrated from 11.1. The other two seem to run fine (CentOS based) but the third one stops after 4min. 30sec. (Ubuntu Server 16.04).

I tried removing the Intel based NIC and replacing it with VirtIO (the other two are still Intel) and I have the same results.

<<update>>
After running for a little while the other two VM's stopped (maybe an hour or so). I did not check the logs and ended up reverting back to 11.1 (all three have ran overnight since).

 

[trevorb]

I didn't notice a signal 10, but did notice in /var/log/middlewared.log
VMService.__init_guest_vmemory():830 - ===> Cannot guarantee memory for guest id: 2

Wired mem sitting at >12Gb of 16Gb
All OK after a reboot, but over time wired mem increases, the VM stops and I can't restart....
Travelling early tomorrow, so will look again over the weekend

 

[KrisBee]

Is this new "memory check" feature in the BETA the culprit?

• Virtual Machines are more crash-resistant. When a guest is started, the amount of available memory is checked and an initialization error will occur if there is insufficient system resources. When a guest is stopped, its resources are returned to the system. In addition, the UEFI boot menu fix allows Linux kernels 4.15 and higher to boot properly.

 

[trevorb]

Is this new "memory check" feature in the BETA the culprit?

Partially.
But the VM will start cleanly after a boot. Then die (haven't been able to check why yet) and not be able to restart due to not enough memory.
Away at the moment and will get to do some more investigation over the weekend.

 

[trevorb]

But the VM will start cleanly after a boot. Then die (haven't been able to check why yet) and not be able to restart due to not enough memory.
Away at the moment and will get to do some more investigation over the weekend.

All good after upgrade to 11B2

 

[trevorb]

All good after upgrade to 11B2

Spoke too soon.
VM had been running for a couple of days, I stopped & started the VM from the GUI as I had removed VNC from the devices, but VM wouldn't start due to 'Cannot guarantee memory for guest'.....
>12GB of my 16GB is showing as wired-memory. No major zfs activity happening.

 

[KrisBee]

Spoke too soon.
VM had been running for a couple of days, I stopped & started the VM from the GUI as I had removed VNC from the devices, but VM wouldn't start due to 'Cannot guarantee memory for guest'.....
>12GB of my 16GB is showing as wired-memory. No major zfs activity happening.

Perhaps I'm being too pessimistic, but I think this new "memory check" feature is going prove troublesome. How much RAM is allocated to this VM? They seem to have introduced a situation where a VM may or may not run depending on the dynamic RAM usage in a running system. Was that always the case?

A user needs to know when guest memory will always be guaranteed and the VM will run - it shouldn't be guess work as it is at the moment. Without a clue as to how this "memory check" has been coded, how is anyone supposed to analyse any problems it might cause?

 

[KrisBee]

The "memory check" code has been added to /usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py


Not that I've made much sense of it at the moment, nor how it plays out when trying to analyse the balance of active, inactive, wired, free and the arc.

 

[trevorb]

How much RAM is allocated to this VM?

3 Gb (box has 16 Gb) + 1 vCPU (of 2 cores)
VM has been stable for a while and have only had these issues with 11 Beta 1 & 2.
I'm sure that the check is showing us that it has a problem, but I'm not sure if it's THE problem.
Closing a VM, should release that RAM back, but it seems to not be available. It could be that the script doesn't adequately calculate what the OS can give a VM. Dunno

 

[trevorb]

Rebooted FreeNAS server and everything came back OK, with VM happily running.
Just over 8 Gb of memory in use and stable for a few hours now.
I'll let it run a little longer to see if things change an then may restart the VM to see what happens....