VM for Home Assistant OS needs multicast

[tprelog]

With USB passthrough finally coming to VMs on SCALE, I was excited to think I could move away from using a docker hack (which ironically seems to be the only thing that actually works as expected, for Home Assistant) to more acceptable setup.

I went ahead and jumped on the Bluefin beta train, got Home Assistant OS installed in a new VM and even managed to add my Aeotec Z-Wave stick. Which works wonderfully, by the way! I was really starting to get excited, this could be the first time ever running Home Assistant in a "fully supported" capacity on TrueNAS.

My excitement was however short lived as I soon found out VMs do not allow multicast traffic, which is a key component for Home Assistant. I found this to be a temporary work around (It was posted in another thread I lost track of).

Code:

ip link set macvtap0 allmulticast on

I thought a init script on startup would be an easy solution but the macvtapX changes with each reboot.

Can someone please point me in direction to get multicast working in my VM?

On side note after I rebooted my SCALE server, I can no longer connect to the VM display either - I just get a 502 Bad Gateway error. The VM still starts and I have ssh access so it's not a show stopper.

 

[tprelog]

I created a bridge and connected the VM to that. It seems everything in Home Assistant is now working as expected! No startup scripts required!

I followed the steps in this post to successfully add the bridge without issue

Can not visit host ip address inside virtual machine

my truenas host ip is 10.31.0.110. my wibdows 10 virtual machine's ip is 10.31.0.251. i could not visit truenas with ip in windows 10 virtual machine. how can i made this happen?

www.truenas.com

 

[gdarends]

Is there a reason for not using the Home Assistant App and using a VM instead?

 

[tprelog]

I just want Home Assistant to work and I'm not going crazy with add-ons. This is all I need. I also want to keep Home Assistant on a seperate IP address. I have a Home Assistant Blue I considered using but it just feels slow after running the HA plugin on my NAS for so many years.

Nothing against the amazing work being done TrueCharts but I feel Home Assistant itself has not been designed to run in a Kubernetes environment. I know there are ways to do things like set specific IP address for Apps and possibly get multicast working but that requires more time learning about Kubernetes and changing load balancers and whatever else is required. By the time you make changes to get Home Assistant working, you've modified the app to the point you fall outside TrueCharts official support.

I don't mean to sound like I'm complaining. I completely understand the focus of SCALE is not single node home users like myself. But I have no time for learning complicated systems like Kubernetes just so I can keep Home Assistant on my NAS.

 

[gdarends]

Question... how did you configure the USB Passthrough.
I am not succeeding with my Conbee II USB Stick

 

[tprelog]

USB passthrough is coming in the SCALE Bluefin release. To be clear, I added my USB while on the last beta release, before the GUI option was finished. I followed the steps in this Jira ticket to add my USB Stick using the API.

To add USB using the GUI, you'll need to be running at least TrueNAS-SCALE-22.12-RC.1

Here's how my looks in the GUI

 

[gdarends]

USB passthrough is coming in the SCALE Bluefin release. To be clear, I added my USB while on the last beta release, before the GUI option was finished. I followed the steps in this Jira ticket to add my USB Stick using the API.

To add USB using the GUI, you'll need to be running at least TrueNAS-SCALE-22.12-RC.1

Here's how my looks in the GUI

View attachment 60135

Yes. I'm wondering if the controller type I'm using is wrong. I used qemu-xhci

 

[mgspivey]

It may be that c210x kernel driver is grabbing the USB device and setting up serial ttyUSBx. Today I updated from the previous train to bluefin RC1 and had a time trying to find out why the dongles were not showing up. Ran dmseg and then saw what was happening, the kernel was grabbing it and setting the ttyUSB devices (udev)

I just set a kernel paremeter to blacklist that driver, rebooted, reconnected the dongles (two) and they were then able to be added to the VM.

went into CLI (actually type cli in a shell on TrueNAS) and then

system advanced update kernel_extra_options="module_blasklist=cp210x"

Doing this allowed me to reboot, power off the hosts and guests.

I also like above use a bridge and haven't had any multicast issues.

 

[mgspivey]

A few typos in my reply, its the cp210x kernel driver. dmesg and the module_blacklist=cp210x". Just wanted to correct them, didnt see an edit.

 

[DaveFace]

I'm having issues running Home Assistant in a VM and curious if it's the same issues you're describing, or something else entirely.

Home Assistant mostly works as expected: I can download stuff from the add-on store, do updates, etc. but accessing most other external services and URL's fails. For example, if I open a terminal (via the SSH & Web Terminal add-on) I can't even ping google.com, but if I open up the VM display from the TrueNAS web UI and use that HA CLI, I can ping it just fine.

I set up a bridge from the above instructions, which was an effort in and of itself, but I still can't ping external URLs.

(Posted a thread on the HA forums but no response there)

 

[tprelog]

That's very odd. I thought the vm display and ssh add-on were basically accessing the same part of the system.

Sorry I don't have any advice to offer at this moment but I just tested mine and seems to work as expected.

Are you using the official ssh add-on or the community one maintained by Frenck?

 

[DaveFace]

After some more poking around, my issue is DNS related. I'm using the SSH & Web Terminal add-on maintained by Frenck, but the official one doesn't work either. Best I can tell, the VM Display (and some services within HA) will use the OS's DNS resolution, but anything in HA that runs under Docker will use the HA DNS service.

Presumably that's why some parts of HA work (like updates), but neither SSH client does. Running ha dns stats shows that the service isn't running at all

Code:

Error: Container hassio_dns is not running

And if I check ha dns logs, something is failing to start with a very helpful error:

Code:

./run: line 5: /usr/bin/coredns: cannot execute binary file: Exec format error
./run: line 5: /usr/bin/coredns: No error information

At the very least I don't think this TrueNas / VM related which narrows it down slightly. Unless it's some very specific routing issue but as yours is working fine, I don't see why that would be the case.

 

[danb35]

./run: line 5: /usr/bin/coredns: cannot execute binary file: Exec format error

That says the coredns binary is in the wrong format for your platform. If you can enter that container and run file /usr/bin/coredns, it may give you more information.

 

[DaveFace]

It's definitely the correct container, at least (amd64) - I tried changing the CPU options in the VM, thinking that it could be mistaking the CPU architecture for something else, but that didn't work. I bit the bullet and set up a new VM, and that one is working just fine. I might just take the hit and pull the configuration across, I've got no idea what went wrong with the other install but it doesn't seem VM related and I doubt I'd get to the bottom of it with the HA devs. Thanks for the help all.

 

[tprelog]

While I was trying to get the QEMU guest agent working in the HAOS VM - I noticed that startup hangs on this job - Sometimes it completes but at least half the time it will just time-out.

Code:

A start job is running for Wait Until Kernel Time Synchronized

What's interesting is that I only have this issue while my VM configured to use a bridge. I discovered this by accident.

While searching for answers about the guest agent, I inadvertently discovered another way to get multicast working is by enabling Trust Guest Filter but this required me to attach the VM directly to my NIC, instead of the using the bridge.

What I noticed is that there is no longer any delay waiting for Kernel Time Sync when attached directly to a NIC.

If I switch back to the bridge the delay comes back. So I solve the multicast and time sync issue but not have access from VM to host. Or I can solve the multicast issue and have access to the host but suffer delays in startup.

Fortunately my motherboard has 2 NICs - I found that attaching the VM to the second NIC, solves the multicast and time sync issue and still has access from VM to host. Of course this is still not ideal (requires a second NIC, extra port on a the switch and sending traffic over the wire to reach the host) but at least I'm getting everything to work. Network traffic between the host and VM is mostly just for backup, so I can live that.

 

[oblivioncth]

While I was trying to get the QEMU guest agent working in the HAOS VM - I noticed that startup hangs on this job - Sometimes it completes but at least half the time it will just time-out.

Code:

A start job is running for Wait Until Kernel Time Synchronized

View attachment 61754


What's interesting is that I only have this issue while my VM configured to use a bridge. I discovered this by accident.

While searching for answers about the guest agent, I inadvertently discovered another way to get multicast working is by enabling Trust Guest Filter but this required me to attach the VM directly to my NIC, instead of the using the bridge.

View attachment 61755


What I noticed is that there is no longer any delay waiting for Kernel Time Sync when attached directly to a NIC.

If I switch back to the bridge the delay comes back. So I solve the multicast and time sync issue but not have access from VM to host. Or I can solve the multicast issue and have access to the host but suffer delays in startup.

Fortunately my motherboard has 2 NICs - I found that attaching the VM to the second NIC, solves the multicast and time sync issue and still has access from VM to host. Of course this is still not ideal (requires a second NIC, extra port on a the switch and sending traffic over the wire to reach the host) but at least I'm getting everything to work. Network traffic between the host and VM is mostly just for backup, so I can live that.

Thanks a ton for providing a good start for the HA portion of the migration of my homelab from ESXi to SCALE.

Dumb question but I just want to make sure I have this straight: Using a bridge with just one NIC allowed Multicast to work (of course with the time sync delay issue)? I was cross-referencing this with your post on the HA forum and just want to be absolutely sure.

The only extra NIC I have is an x4 card and the only slot I have free at the moment is a 1x slot. I could order another NIC, but that seems a little silly just for this, especially given it incurs the downsides of the extra cable/switch port.

I think for the moment I can deal with the startup delay.

 

[tprelog]

Using a bridge with just one NIC allowed Multicast to work (of course with the time sync delay issue)?

Yep, using a bridge everything multicast should work, as far as I can tell anyways.

I was cross-referencing this with your post on the HA forum

In case anyone else was looking for help getting HA installed.

Installing HAOS in a VM on TrueNAS SCALE

Install Home Assistant OS Begin in the TrueNAS UI by adding a new ZVOL that is at least 32 GiB in size. Note the ZVOL path. Then, using the TrueNAS shell, run these commands to (1) download, (2) extract, and (3) write the HAOS image for KVM to the ZVOL you created. In the 3rd command, replace...

community.home-assistant.io