[How-To] A guide to migrate FreeBSD VM from ESXi to FreeNAS (bhyve)

[Myka]

This Resource is available at:
https://forums.freenas.org/index.ph...ate-freebsd-vm-from-esxi-to-freenas-bhyve.72/

The original text is below. It may be out of date.

Show : Original Resource text

I moved my FreeBSD VM (v7) from ESXi 5.5.0 to bhyve under FreeNAS-11-MASTER-201709210509. Below are short instructions with my example.

1. Prepare VM on ESXi. With your preferred method:
   - shutdown VM
   - remove all snapshots
   - remove from ESXi inventory to avoid problems. Do not delete data files
   
2. Copy ESXi VM virtual disk files. Mine had 5 virtual disks attached
   - ssh root@esxihost
   - scp -r /vmfs/volumes/datastore2/srv_10 root@freenas:/mnt/tank/vm
3. Convert. I used jail under same FreeNAS box for this
   - create a standard type jail. I named it VirtualBoxJail. Do not confuse it with legacy VirtualBox template
   
   
   
   - add storage with a copy of ESXi virtual disks
   
   
   
   - start this jail and install VirtualBox inside it using shell
   pkg update
pkg install virtualbox-ose-nox11
   - convert all vmdk hard drive files to raw image files
   cd /mnt/srv_10/
VBoxManage clonehd srv_10.vmdk srv_10_bhyve_volume_1.raw --format RAW
VBoxManage clonehd srv_10_1.vmdk srv_10_bhyve_volume_2.raw --format RAW
...

   - create ZVOLs for each VM disk same size like original + one 100 MB for EFI. We will need it to fix VM boot
   
   
   
   - copy data from RAW files to ZVOLs repectively. I added a pipe via pv to see some progress but you can ommit that
   dd if=/mnt/tank/vm/srv_10/srv_10_bhyve_volume_1.raw | pv -petr | dd of=/dev/zvol/tank/vm/srv_10_vol_1
dd if=/mnt/tank/vm/srv_10/srv_10_bhyve_volume_2.raw | pv -petr | dd of=/dev/zvol/tank/vm/srv_10_vol_2
dd if=/mnt/tank/vm/srv_10/srv_10_bhyve_volume_3.raw | pv -petr | dd of=/dev/zvol/tank/vm/srv_10_vol_3
dd if=/mnt/tank/vm/srv_10/srv_10_bhyve_volume_4.raw | pv -petr | dd of=/dev/zvol/tank/vm/srv_10_vol_4
dd if=/mnt/tank/vm/srv_10/srv_10_bhyve_volume_5.raw | pv -petr | dd of=/dev/zvol/tank/vm/srv_10_vol_5
4. Assemble new bhyve VM
   - create new VM
   - configure nic
   - add virtual CD. You will need to download and mount FreeBSD image on it to fix EFI boot
   curl https://download.freebsd.org/ftp/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-disc1.iso
   - add earlier created ZVOLs (EFI and all other VM disks)
   
   
5. Fix EFI , mounting points and NIC names
   - start VM. It should boot from attached FreeBSD iso. Start VNC session and Select Live CD. Now we will create EFI partition with image files.
   - login with root. Check virtual disk names
   dmesg | grep ada
   - on that EFI 100 MB disk create GUID partition table
   gpart create -s GPT /dev/ada0
   - create a GUID Partition Table (GPT) scheme:
   gpart create -s GPT /dev/ada0
   - Add the EFI partition with the label "EFI" aligned to 1MB:
   gpart add -t efi -a 1M -l EFI /dev/ada0
   - create the FAT16 filesystem for the EFI partition, with a label of FreeBSD_EFI:
   newfs_msdos -F 16 -L FreeBSD_EFI /dev/ada0p1
   - mount it:
   mount -t msdosfs /dev/ada0p1 /mnt
   - create the directory for the FreeBSD UEFI bootloader:
   mkdir -p /mnt/efi/EFI/BOOT
   - copy the FreeBSD UEFI first-stage bootloader into that directory with the name which firmware expects:
   cp /boot/boot1.efi /mnt/efi/EFI/BOOT/BOOTX64.EFI
   - unmount the filesystem:
   umount /mnt
   - now mount your root (/) and edit your fstab as your drive names must have changed. Alternatively you can fix nic name in rc.local
   mount /dev/ada1p2 /mnt
vi /mnt/etc/fstab
vi /mnt/etc/rc.conf
umount /mnt
   
   my changes were
   da0p2 -> ada1p2
   da1p1 -> ada2p1
   da2p1 -> ada3p1
   da3p1 -> ada4p1
   da4p1 -> ada5p1
   vmx0 -> vtnet0
   
6. After all this is done remove CD from VM and reboot it. You should be good to go

p.s. You can remove vmdk, VirtualBox converted raw files and VirtualBoxJail. Now VM data lives inside ZVOLs

used material from
https://forums.freebsd.org/threads/51393/
http://justinholcomb.me/blog/2016/03/26/migration-to-freebsd-part4.html
m0nkey_ thanks for comments

 

[Ericloewe]

This is probably useful to many people, so it'd be a good idea to post this in the resources section. Once you've done that, ping me here so that I can assign this thread as the discussion thread.

 

[m0nkey_]

create VirtualBoxJail

You might want to differentiate between creating the jail using the legacy VirtualBox template and a standard template, since the VirtualBox template is deprecated.

 

[Myka]

This is probably useful to many people, so it'd be a good idea to post this in the resources section. Once you've done that, ping me here so that I can assign this thread as the discussion thread.

@Ericloewe
I that is addressed to me I would need a hint how to it.

@m0nkey_
Thanks for comment. Seems like there could be some confusion. I did an update.

 

[Ericloewe]

I that is addressed to me I would need a hint how to it.

https://forums.freenas.org/index.php?resources/

Use the Add Resource button, choose Plugins and Jails, copy/paste your document into the Resource and choose "Does not have a file".

 

[Stux]

Use the Add Resource button, choose Plugins and Jails, copy/paste your document into the Resource and choose "Does not have a file".

Maybe you should make a resource about making a resource from a topic ;)

 

[Ericloewe]

Maybe you should make a resource about making a resource from a topic ;)

The problem is one of discoverability. By the time anyone has opened the resource on how to make a resource, they've probably figured out how to make one.

 

[Stux]

The problem is one of discoverability. By the time anyone has opened the resource on how to make a resource, they've probably figured out how to make one.

It's more about the bit where you msg @Ericloewe to link the old thread to the new resource ;)

 

[Ericloewe]

I get notified either way, since I'm subscribed to all resource categories, so I'm not too fussed about that.

 

[Myka]

The problem is one of discoverability. By the time anyone has opened the resource on how to make a resource, they've probably figured out how to make one.

I need another hint. Here is my screenshot. It seems that is really hard for me.

 

[Myka]

I need another hint. Here is my screenshot. It seems that is really hard for me.

I see this is fixed now. Add Resource button is now in place where it should be. I reposted article there. In my view you can remove posts about resource button as they are not related to subject.[/QUOTE]

 

[Ericloewe]

I'll just lock this thread instead since it's just metadiscussion.