SOLVED Bhyve VMs: unable to VNC in, stopping when attempting to connect

[Gruselbauer]

Hello everyone,

I've set up a FreeNAS install for a business venture with my best friend. Mostly needed was reliable, enterprise grade storage on the cheap, and for that I'm just hella thankful to everyone involved in FreeNAS. A 12tb NAS with the option for two drives failing simultaneously at below 600eur is still blowing my mind.

Basically, I have the exact problem somebody detailed in this thread:
https://linustechtips.com/main/topic/948913-how-to-vnc-connect-to-a-freenas-vm/
Hardware is as follows:
2x Intel Xeon E5506
72gb DDR3 ECC Memory
7x 2tb spinning rust (different brands and lines) in RAIDZ-2
2x 120gb SSD mirrored as system disks
No name server board off Ebay

I know I am limited to one vCPU by the physical CPU, also the FreeNAS web GUI tells me that. Everything else works flawlessly, from network shares to jails and plugins. Trying to create a Windows or Linux VM however just does not work no matter what option I change. I've toyed with different NIC devices, changed the UEFI setting to UEFI_CSM, allocated more or less memory and tried my luck with an array of different ISOs from Ubuntu Server to Windows 10 in both 32 and 64 bit flavors.

When I uncheck "Wait to Boot" in the VNC device options, the VMs won't start at all and give me this error:

Code:

Error: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 161, in call_method
    result = await self.middleware.call_method(self, message)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1109, in call_method
    return await self._call(message['method'], serviceobj, methodobj, params, app=app, io_thread=False)
  File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1049, in _call
    return await methodobj(*args)
  File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 664, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 1132, in start
    await self._manager.start(vm)
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 61, in start
    list(done)[0].result()
  File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py", line 315, in run
    raise CallError(f'VM {self.vm["name"]} failed to start: {output}')
middlewared.service_exception.CallError: [EFAULT] VM blaaaa failed to start: 04/12/2018 19:17:10 Listening for VNC connections on TCP port 5857
04/12/2018 19:17:10 Listening for VNC connections on TCP6 port 5857
ROM boot failed: unrestricted guest capability not available
fbuf frame buffer base: 0x942e00000 [sz 16777216]

When I check that box and try to connect via the web interface, the popup for http://myinternalip:port/vnc.html?autoconnect=1 gives me error number 1006 and then the VM stops running. Also note that the port number it tries to access is wrong, and always by exactly 100. If I have VNC configured to run on 13199, the web interface will try to open 13099 and so on, reliably and no matter. If I try to connect with a Linux/macOS/Windows VNC client, I get "unexpected connection error" replies, and after the first attempt the VM will also stop running.

I realize bhyve is a rather young project and I'm running Xeons from 2009, so I'm pretty much prepared to hear that this is a hardware incompatibility. Still, any help would be immensely appreciated and maybe I can be of use in mitigating this problem? Not much data on the NAS so far and I'm down to reinstall or try a more recent build.

Thanks a lot for any replies!

 

[dlavigne]

Which FreeNAS build version (from System -> Information or the Dashboard)?

 

[Gruselbauer]

FreeNAS-11.2-RC2
(Build Date: Nov 15, 2018 20:21)

Thanks for replying :)

 

[guermantes]

https://forums.freenas.org/index.php?threads/howto-how-to-boot-linux-vms-using-uefi.54039/

Sorry if I don't analyse your log in detail, but might this be your problem?

 

[KrisBee]

This line is your traceback:

"ROM boot failed: unrestricted guest capability not available"

Have you checked your CPU is supported by bhyve? See guide at https://www.ixsystems.com/documentation/freenas/11.2/virtualmachines.html

"To verify that an Intel processor has the required features, use Shell to run grep VT-x /var/run/dmesg.boot. If the EPT and UG features are shown, this processor can be used with bhyve."

Also,

Code:

root@freenaskvm:~ # grep unrestricted /usr/local/lib/python3.6/site-packages/middlewared/plugins/vm.py

        ug = sysctl.filter('hw.vmm.vmx.cap.unrestricted_guest')
        data['unrestricted_guest'] = True if ug and ug[0].value else False

 

[Gruselbauer]

@guermantes Thanks for the reply, but sadly not. I can't even start / VNC into machines.

@KrisBee

Code:

[root@metaverse ~]# grep VT-x /var/run/dmesg.boot
  VT-x: PAT,HLT,MTF,PAUSE,EPT,VPID

So, I'm missing unrestricted guest mode, and hence can't use bhyve, yes? Damn. I had pondered running Proxmox as the host operating system and virtualising FreeNAS with PCI passthrough of the LSI HBA in the machine. But that's a hassle and somehow gave a less than stable impression. After all, I'd rather FreeNAS do its thing as sound as possible than have virtual machine support.

KVM/QEMU don't seem to mind though. Maybe there is a chance that future updates of Bhyve will do without UG?

Oh, another thing that comes to mind: I can connect to the virtual machines via the Serial option of the FreeNAS webinterface, and it says "Connected", but there's no further output from the install media.

Anyone know of an ISO / live cd with SSH open per default? I'd like to try if the machines actually do keep running until that first VNC connection attempt. Which is what the web interface tells me.

Edit: I think I'm going to pick up two 32nm Xeons on the cheap. The X5650 is twenty bucks a piece on eBay and according to my Googlings supports unrestricted guests. Going to search around a bit for the best LGA1366 CPU I can find, see if it works then.

Edit2: 2x X5670 coming by the end of the week.
http://www.tomshardware.co.uk/answers/id-3355611/1366-cpu-family-question.html confirms, but if someone here is running FreeNAS VMs on a Westmere CPU with no trouble and wants to reassure me, I'd be extra grateful.

Thanks again for the replies folks. Lovely community :3

 

[Gruselbauer]

Double post to update: Problem solved, it was indeed a CPU problem. Updating to the X5670 and now Bhyve works like a charm. Thank you for your help everyone!

 

[flingwing]

bummer, I think I'm running into the same thing... my CPU doesn't support VM's.

 

[chris4nas]

bummer, I think I'm running into the same thing... my CPU doesn't support VM's.

VT-x had to be enabled in my BIOS.