VM won't boot if more than 1 vCPU

[tommyatkadx]

I'm working with the latest FreeNAS 11.1 and trying to play with bhyve.

Installing Windows Server 2016.

When I assign only 1 vCPU to the VM, everything goes well.

If I set more than 1 vCPU, VM will sometimes boot, but most of the time, freeze on boot. Been waiting for hours and the VM is still not up.

If the VM doesn't get up after 3-5 minutes, I restart the VM, until I can get the VM to boot. When the VM finally get to boot, everything looks to be working...

How can I troubleshoot this?

 

[mjt5282]

older versions of windows would malfunction when you changed the virtual number of CPUs after installation, because the kernel files installed would be different. I had to re-install Windows after adding a second CPU . A big virtual pain, and WS 2016 maybe similar.

 

[tommyatkadx]

I have the same issue to boot on the Windows installation DVD. When only 1 vCPU, it boot #1, if I set 2 or more vCPU, it will loop on the DVD for hours without success.

This is not related to # of vCPU in Windows, Windows Server 2016 support it 100%

 

[wil]

Does your host CPU support Unrestricted Guest?

eg.
grep VT-x /var/run/dmesg.boot
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID

There should be "UG" in there for bhyve to actually allow you to use more as one 1 vCPU.

 

[tommyatkadx]

This is opteron CPU and not intel, and based on: https://www.freebsd.org/doc/handbook/virtualization-host-bhyve.html POPCNT needs to be on Features2=, and it is.

root@freenas:/nonexistent # grep VT-x /var/run/dmesg.boot
root@freenas:/nonexistent # grep POPCNT /var/run/dmesg.boot
Features2=0x1e98220b<SSE3,PCLMULQDQ,MON,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX>
root@freenas:/nonexistent #

Any other idea?

 

[wil]

Is your NIC set to virtio hardware and does it use the latest virtio drivers? The e1000 NIC that comes with the default causes a number of strange issues including hanging issues.

 

[tommyatkadx]

when using the virtio drivers, Windows won't recognise the NIC, so, I'm using the e1000.

Removing the NIC won't help the VM booting when more than 1 vCPU

 

[madmac]

If you INSTALL Windows with only 1 vCPU, Windows will install the single-CPU version of the Hardware Abstraction Layer (HAL). Adding a 2nd vCPU will make this VM unstable, as the wrong HAL is installed. If you had multiple vCPUs set when you did the install, then this is NOT the issue . . .

I had network issues with the Intel E1000 drivers. When I switched the bhyve config to VirtIO, and booted, I of course, had no network connectivity. VNC got me into the console, and I was able to use to the Device Manger to browse for and install the downloaded VirtIO drivers (as an .ISO) I had placed on a USB Key and Imported and mapped to the CD-ROM drive in the bhyve configuration. This resolved my Networking issue.

If you're successfully running the VirtIO Networking drivers, and you did a Multi-CPU install so that you have the right HAL, I'm out of ideas . . .

Doug