Bhyve VM PCI Passthrough - Won't boot (Previously working)

Dekkars

Dabbler
Joined
May 18, 2017
Messages
13
Hi All,

I had a USB PCI passthrough to a VM that was previously working without a hiccup, and ever since a reboot an hour ago the VM will not start if the PCI passthrough is loaded as a device.

If I remove the PCI passthrough as a VM device everything works just fine.

Any thoughts?
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
How did you set up the passthrough? Did you edit loader.conf or did you set a loader tunable under System->Tunables?
 

Dekkars

Dabbler
Joined
May 18, 2017
Messages
13
How did you set up the passthrough? Did you edit loader.conf or did you set a loader tunable under System->Tunables?

I went through tunables. This has survived multiple restarts and only now is seemingly deciding that it won't work...

When I run pciconf -lv I still see it listed there too...
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
This might be a timing issue, then. Try setting the VM not to auto-start, and reboot again. See if the VM starts manually.
 

Dekkars

Dabbler
Joined
May 18, 2017
Messages
13
The VM was never set to auto-boot. I even tried telling it to wait for VNC to connect before starting.

If the PCI device is enabled the VM refuses to start at all.

Tried creating a new VM too. No luck.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
OK, we'll need to see what is acting up. Try looking in /var/log/middlewared.log* for any lines including bhyve.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,358
I am seeing a similar thing.

If I have a pfSense VM with two PCI pass-through NICs, then if I do a guest based restart (ie tell pfsense to reboot, or an upgrade etc) it won't reboot. It does shutdown though.

You can see the bhyve process is no longer present, but TrueNAS still thinks the VM is running. Pressing "stop" in TrueNAS will immediatley detect it as stopped, and thus it can be started.

If I remove the pass-through devices (and added some virtual devices), then it works correctly.

Doing a host side restart works fine. Ie restart/stop in the TrueNAS GUI. And autoboot after TrueNAS reboot also works fine.
 
Top