Advanced VM configuration editing

[skittlebrau]

Loving all the work you all have been doing to get TrueNAS SCALE to final release!

After playing around with the latest alpha release, is it possible for edits made with virsh on the command line to 'stick' and not be overwritten when starting/stopping VMs from the UI (or starting a VNC session) or is it unavoidable?

Say I wanted to make modifications that are currently not possible from the UI like use a specific custom topology, direct boot from NVME devices that have been passed through, specify Q35 model version etc, would I be better off ignoring the UI and just interacting with VMs with virsh via CLI?

 

[morganL]

The issue is how does TrueNAS ensure that all your configurations survive reboots and software updates? If its done through APIs (or WebUI) then TrueNAS has all the configuration data. If you bypass that, then on a reboot or upgrade it may not survive.

So, suggestion would be to define and prioritize the things you would need for your use case.... join the developers, write up your needs or make a suggestion in our bug tracker.

At a high level, we are looking to make it possible to template VMs so it is easy to create VMs that are similar. In this case, we would like to see what options should be configurable in the templates.

 

[skittlebrau]

Thanks, I think it'll be worth submitting it as feedback so I'll go do that.

I think there'll be many people out there who would love to be able to edit the libvirt XML file directly to make changes to their VM, similar to what you can do on other distros. Perhaps if TrueNAS detected custom changes that have been made to the VM's XML, it could at least 'lock' the editing options for that VM and notify the user that any further changes must be made manually.

 

[silverback]

Loving all the work you all have been doing to get TrueNAS SCALE to final release!

After playing around with the latest alpha release, is it possible for edits made with virsh on the command line to 'stick' and not be overwritten when starting/stopping VMs from the UI (or starting a VNC session) or is it unavoidable?

Say I wanted to make modifications that are currently not possible from the UI like use a specific custom topology, direct boot from NVME devices that have been passed through, specify Q35 model version etc, would I be better off ignoring the UI and just interacting with VMs with virsh via CLI?

Currently it is necessary for me to build the VM on a Debian machine and import the VM onto Scale with virsh. You may need to add virsh commands as “post ini” tasks as well, depending on what you require.

 

[mvipe01]

It would be nice to have a custom box for adding custom devices or passthrough shares such as:
<filesystem type='mount' accessmode='passthrough'>
<source dir='/mnt/data1/backups'/>
<target dir='backups'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/>
</filesystem>

 

[mgoulet65]

My use case would be satisfied if we could create a VM in Scale using an existing qcow2 residing in a pool as the boot device. Is that in the roadmap?

 

[morganL]

My use case would be satisfied if we could create a VM in Scale using an existing qcow2 residing in a pool as the boot device. Is that in the roadmap?

Best to describe the use case and why QCOW2 is better than a ZVOL.. what are the saving/benefits? Make it a "suggestion" in "report-a-bug" and others can up-vote it with their needs.

It should be noted that using ZVOLs enables snapshots,replication and cloning. Using QCOW2 files would remove that option except at the dataset level.

 

[mvipe01]

Best to describe the use case and why QCOW2 is better than a ZVOL.. what are the saving/benefits? Make it a "suggestion" in "report-a-bug" and others can up-vote it with their needs.

It should be noted that using ZVOLs enables snapshots,replication and cloning. Using QCOW2 files would remove that option except at the dataset level.

Why not support both? Afaik its already supported by libvirt and why not add as many options as possible? It is far easier to simply transfer an existing qcow image especially if migrating from other setups such as base libvirt or unraid. Not to mention its just simple. Some people would choose the simplicity over potential ability.

It is the same as my idea for the folder passthrough, its already supported by libvirt but we are unable to use it because there is no option to do so in the device tab. Instead I have to use NFS to mount.

I personally moved over from unraid and had several issues with converting the image and then dd'ing to the zvol. It worked great on all my linux vm but every single windows vm ended up with issues rebooting or pausing or plain crashing when converting and imaging them over. I had to reinstall of the windows VM.

 

[mgoulet65]

Why not support both? Afaik its already supported by libvirt and why not add as many options as possible? It is far easier to simply transfer an existing qcow image especially if migrating from other setups such as base libvirt or unraid. Not to mention its just simple. Some people would choose the simplicity over potential ability.

It is the same as my idea for the folder passthrough, its already supported by libvirt but we are unable to use it because there is no option to do so in the device tab. Instead I have to use NFS to mount.

I personally moved over from unraid and had several issues with converting the image and then dd'ing to the zvol. It worked great on all my linux vm but every single windows vm ended up with issues rebooting or pausing or plain crashing when converting and imaging them over. I had to reinstall of the windows VM.

It is Windows VMs that are giving me fits as well. I can qemu-img convert them to qcow2 asnd run them all day in virsh on Ubuntu. But I have yet to get TrueNas to successfully boot one following all of the How Tos to convert to raw and dd to the zvol.

 

[Michael Schefczyk]

A strong argument for using qcow2 is that live backups (-> live full disk backup https://libvirt.org/kbase/live_full_disk_backup.html) are possible. Such backups create a separate full qcow2-file which can be handled in a flexible manner (e.g, revert to a previous version by starting from that backup file or migrate to a different KVM system).

I am not certain how to manage that with a zvol in a way that there will be a consistent state backup from a running VM. If that is possible, it would be great if someone could state that and ideally document how to do it.

 

[mgoulet65]

Why not support both? Afaik its already supported by libvirt and why not add as many options as possible? It is far easier to simply transfer an existing qcow image especially if migrating from other setups such as base libvirt or unraid. Not to mention its just simple. Some people would choose the simplicity over potential ability.

It is the same as my idea for the folder passthrough, its already supported by libvirt but we are unable to use it because there is no option to do so in the device tab. Instead I have to use NFS to mount.

I personally moved over from unraid and had several issues with converting the image and then dd'ing to the zvol. It worked great on all my linux vm but every single windows vm ended up with issues rebooting or pausing or plain crashing when converting and imaging them over. I had to reinstall of the windows VM.

After much fiddling, I was able to get a Windows VM converted over. I'd still prefer to be able to use qcow2.

 

[thetomcorp]

I really like TrueNAS scale so far. I've been using TrueNAS (FreeNAS) for over 10 years. I've been running my Storage server for storing EVERYTHING, and using a second Virtualization Server to host appications/VMs, pretty much using raw KVM on Fedora Linux. I was excited to see that KVM came to TrueNAS Scale.

qcow2 being a standard for KVM I want to stick with that. I see this Jira ticket didn't get any activity https://ixsystems.atlassian.net/browse/NAS-115656 so my assumption is qcow2 support isn't coming to TrueNAS scale anytime soon.

If I want a consolidated environment I'd have to either do Nested virtualization in TrueNAS Scale, or Virtualized TrueNAS Scale on Fedora. Since I have the extra server I'm going to try both, but in the end I'll probably just stick with 2 separate servers.