Virtualbox jail not running after virtualbox-ose-kmod updated

[Kam]

Hi all,
as part of Feature #18630 Update to 2016Q4 Ports Tree is updated virtualbox-ose-kmod from 4.3.34 to 5.1.6:
current night build FreeNAS-9.10-MASTER-201612100422 (4d7d293)

Code:

[root@freenas3] ~# pkg info | grep -i virtualbox
virtualbox-ose-kmod-5.1.6  VirtualBox kernel module for FreeBSD

last stable FreeNAS-9.10.1-U4 (ec9a7d3)

Code:

[root@freenas] ~# pkg info |grep virtualbox
virtualbox-ose-kmod-4.3.34  VirtualBox kernel module for FreeBSD

Because of this update startup of Virtualbox VM in virtualbox jail fails and in dmesg output is error:

Code:

SUP_IOCTL_COOKIE: Version mismatch. Requested: 0x1a0007  Min: 0x1a0005  Current: 0x260000

Will you create new virtualbox jail template with 5.1.6 vbox version?

With 9.10 are many users using virtualbox jail and FreeNAS 10 is not yet out.

If not, here are steps how to create new virtualbox jail yourself, for those, who want to still use virtualbox with current 9.10 version after update of virtualbox-ose-kmod to 5.1.6.

Before that, backup your machines in original virtualbox jail, so that you can move them to new jail after creation.

Steps to create new virtualbox jail
===================================
Jails -> Add Jail -> "virtualbox"
- this will create standart freebsd jail, do not choose old vbox template

jexec virtualbox

vi /usr/local/etc/pkg/repos/FreeBSD.conf
- url: "pkg+http://pkg.FreeBSD.org/freebsd:10:x86:64/latest",
+ url: "pkg+http://pkg.FreeBSD.org/freebsd:10:x86:64/quarterly",

pkg upgrade
pkg install virtualbox-ose phpvirtualbox apache24 mod_php56

adduser

Code:

Username: vbox
Full name: vbox
Uid (Leave empty for default): 1001
Login group [vbox]:
Login group is vbox. Invite vbox into other groups? []: vboxusers
Login class [default]:
Shell (sh csh tcsh git-shell nologin) [sh]:
Home directory [/home/vbox]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password:VBox123
Enter password again:VBox123
Lock out the account after creation? [no]:
Username  : vbox
Password  : *****
Full Name  : vbox
Uid  : 1001
Class  :
Groups  : vbox vboxusers
Home  : /home/vbox
Home Mode  :
Shell  : /bin/sh
Locked  : no
OK? (yes/no): y
adduser: INFO: Successfully added (vbox) to the user database.
Add another user? (yes/no): no
Goodbye!

vi /etc/rc.conf

Code:

+ apache24_enable="YES"
+ vboxwebsrv_user="vbox"
+ vboxwebsrv_enable="YES"

service vboxwebsrv start

vi /usr/local/www/phpvirtualbox/config.php

Code:

- var $password = 'pass';
+ var $password = 'VBox123';

vi /usr/local/etc/apache24/httpd.conf

Code:

- DocumentRoot "/usr/local/www/apache24/data"
+ DocumentRoot "/usr/local/www/phpvirtualbox"
- <Directory "/usr/local/www/apache24/data">
+ <Directory "/usr/local/www/phpvirtualbox">

# In section <IfModule mime_module>, add after x-gzip type for php module:
  AddType application/x-compress .Z
  AddType application/x-gzip .gz .tgz
#
+  AddType application/x-httpd-php .php
+  AddType application/x-httpd-php-sources .phps
#

# workaround for older phpvirtualbox-5.0.5
vi /usr/local/www/phpvirtualbox/endpoints/api.php

Code:

-  $response['data']['responseData']['phpvboxver'] = @constant('PHPVBOX_VER');
+  $response['data']['responseData']['phpvboxver'] = "5.1-0";

service apache24 start

ifconfig -a|grep inet

http://x.x.x.x
admin/admin

Enjoy!

I would create also template from this, but do now know how..
Kam

 

[JohnDigital]

Great info. Ill give this a try.

 

[JohnDigital]

Got around to tinkering with this tonight. Works great. Had to tinker around a bit, and very stupid of me I think everything I did and redid was not necessary. Seems my old info was stored in Chrome, and in the end it fired right up after I cleared the cache. Before I blew the jail away I exported my machine(s) as .ova appliances and imported after the reinstall. Everything is up and running just fine. Great tutorial Kam. My instinct says lots of people are going to be searching for this over the next few weeks. It worked for me. Thanks again.

 

[picklefish]

What does this mean?
"Before that, backup your machines in original virtualbox jail, so that you can move them to new jail after creation."

 

[Kam]

Machines are saved in virtualbox jail under "/home/vbox/VirtualBox VMs" folder.
That folder is accessible also from freenas host under
/mnt/<pool name>/jails/<virtualbox vm name>/home/vbox/VirtualBox VMs

So you can copy contents of this folder somewhere outside of jail and then copy back to new virtualbox jail. After that add machine back under new virtualbox version using

jexec virtualbox
su - vbox VBoxManage registervm /home/vbox/VirtualBox\ VMs/<machine>/<machine>.vbox

 

[picklefish]

su - vbox VBoxManage registervm /home/vbox/VirtualBox\ VMs/<machine>/<machine>.vbox
I get cannot open VBoxManage: No such file or directory

EDIT: Ahah, I think it needed -c argument so
su - vbox -c "VBoxManage registervm /home/vbox/VirtualBox\ VMs/<machine>/<machine>.vbox"

I also had to chown the folder: chown -R vbox:vbox /home/vbox/VirtualBox\ VMs/<machine>/

 

[JohnDigital]

I dont use VB very extensively. I only run a Musicbrainz VM. Better wait for Kam to specify.

Before I did anything I made backups of my musicbrainz-vm by exporting appliance, copying that resulting .ova (it will put it in the /usr/home/vbox/.config/ folder) file to a directory outside of the jail, doing Kams install method and then importing that appliance via the .ova file. Once I cleared my cache it fired up no issues. Not sure if this will work for you.

 

[Kam]

Thanks John for info about need to clear a cache. I had now same problems on other machine and once the cache in Firefox was cleared it started to work!
Regarding registration picklefish asked - sorry for incomplete syntax. I used su - vbox firsr and then run the register command, but here I put it on one line. Thanks for this additional info.

What is still unclear to me is position of freenas developers if they are ok with such own virtualbox jail setup, or one day if they stop including needed kmod drivers together with freenas kernel. Because then this solution will not work. I saw only announcement that they are stopping support of very old virtualbox template.

One more info - I think with later updates may again virtualbox stop working if freenas host gets updated. That is because instalation in jail depends on host support of this package:

[root@freenas] ~# pkg info|grep virtualbox
virtualbox-ose-kmod-5.1.6 VirtualBox kernel module for FreeBSD

If this gets newer version, vm startup may again have problem. I hope that it could be later fixed by upgrading jail packages using:

jexec virtualbox
pkg search virtualbox
-> check if virtualbox packages with same version exists, which are shipped with freenas kernel. if yes then
pkg upgrade

if are not same version, we would need to find them somewhere else.
for example currently is latest 5.1.10 in freebsd latest, that's why I switched repo to quarterly, where was older 5.1.6 version.

 

[Sillyus Sodus]

I can't seem to get the graceful savestate shutdown and autostart to work any more. Are all the vboxheadless statements the same as before?

e.g. vboxheadless_machinename_stop="savestate"

 

[picklefish]

I agree Kam, I'm really surprised that so few people have commented here given that it just breaks vbox. I think people have stopped updating because things break every time.

 

[Mal de Mer]

I noticed this in release notes for 9.10.2 "FreeBSD Jails now default to a FreeBSD 10.3-RC2 based template. Old jails, or systems on which jails have been installed, will still default to the previous FreeBSD 9.3 based template. Only those machines using jails for the first time (or deleting and recreating their jails dataset) will use the new template."

So I booted into 9.10.1-u4 to make the VBox based VM that I needed. It worked without any issues, and then I booted back to 9.10.2 and the VM still runs without a problem. That's the first time that I have used the saved boot environment feature.

 

[Dtime]

Dang. I have two dozen virtual machines defined in vBox. 4 of those VMs are set to savestate by editing jails/.VirtualB_1.meta/jail-pre-stop with the required commands. Also using /etc/rc.conf from within vBox jail to autostart those VMs. Everything was working fine with freenas 9.10.1-U4. Upgrading to 9.10.2 broke the autostart for me.

Looks like I might have to stay on 9.10.1.

 

[Mal de Mer]

Earlier today I made the changes in /etc/rc.conf to use autostart. It does seem to be broken in 9.10.2. My VM runs correctly when started manually but always aborts when fired up through autostart. Maybe the changes to jails in 9.10.2 didn't get much QA time, or we're being forced to bhyve despite the GUI features being missing at present (at least I couldn't get a GUI to work, but in my defence I'm not a FreeBSD guy).

Autostart still seems to work on 9.10.1-u4. I did have to reinstall though - just going back using the 9.10.1 boot environment wasn't sufficient. Fortunately, a reinstall, config reload and reinstalling the VM doesn't take much time.

 

[karlinhigh]

Thank you so much, Kam! I was trying to figure this out on my own, with no success. FreeNAS admins, is there any chance something like this could be added to Chapter 13 Jails in doc.freenas.org? At least until the new bhyve system is as capable and usable as the old VirtualBox template?

 

[Jailer]

FreeNAS admins, is there any chance something like this could be added to Chapter 13 Jails in doc.freenas.org? At least until the new bhyve system is as capable and usable as the old VirtualBox template?

Not gonna happen. FreeNAS 9 is feature locked and will only receive security and bug fixes going forward.

 

[karlinhigh]

Not gonna happen. FreeNAS 9 is feature locked and will only receive security and bug fixes going forward.

So they can't even update the documentation?

I can understand not adding features, but if the docs can include links to YouTube videos with iohyve tutorials, surely they could have a link to one about re-creating a VirtualBox setup?

 

[jkh]

I can't speak to the documentation issue (please file a ticket, don't just ask on the Forum - it's unlikely that any docs people will even read this thread, but they DO read their tickets).

Regarding the virtualibox issue from a technical perspective, it was always clear to us that we were going to have a problem with the virtualbox jail when we added it, because the jail contents and the kernel / kernel module ABI are rev-locked and *not* updated at the same time. They can't be, because their lifecycle management is totally different.

That is why the virtualbox jails are being deprecated now that there is an alternative - it's just too hard to make them not break with each and every release that updates the kernel or kernel modules.

 

[Sillyus Sodus]

Mine seems to autostart fine in 9.10.2 but it doesn't stop via the save state method. I don't think it even sends a shutdown to the VM.

 

[Dtime]

@Sillyus Sodus

Mine seems to autostart fine in 9.10.2 but it doesn't stop via the save state method. I don't think it even sends a shutdown to the VM.

Save state never did work for me using vboxheadless. I created script called autostopvms.sh inside the virtualbox jail that contains lines similar to the following for each VM to be stopped:

Code:

su -l vbox -c "VBoxManage controlvm testVM savestate --type headless"

After that script was created you can edit the jail-pre-stop script to call autostopvms.sh. I added the following 2 lines to jail-pre-stop

Code:

jexec VirtualBox_1 /autostopvms.sh  
sleep 20

You can find jail-pre-stop in the directory : /mnt/tank/jails/.[JailName].meta

Please note: I am on freenas 9.10.1

 

[Sillyus Sodus]

Nice one, this works well. I called the su command directly with jexec in the jail-pre-stop script.

Code:

jexec virtualbox_1 su - vbox -c "vboxmanage controlvm vmname savestate --type headless"

Is the sleep command necessary?

Also, how did you know the name of my jail? ;)

@Sillyus Sodus


Save state never did work for me using vboxheadless. I created script called autostopvms.sh inside the virtualbox jail that contains lines similar to the following for each VM to be stopped:

Code:

su -l vbox -c "VBoxManage controlvm testVM savestate --type headless"

After that script was created you can edit the jail-pre-stop script to call autostopvms.sh. I added the following 2 lines to jail-pre-stop

Code:

jexec VirtualBox_1 /autostopvms.sh 
sleep 20

You can find jail-pre-stop in the directory : /mnt/tank/jails/.[JailName].meta

Please note: I am on freenas 9.10.1