Trying to advertise other hosts with Avahi, but config is cleared after reboot.

[ITOperative]

Hey all,

So I wanted to advertise several hosts on my network via Avahi, and since TrueNAS has it installed (and its storage supports other VMs on the parent host), I was hoping to use TrueNAS' instance of Avahi to do the advertising.

I edited /etc/local/avahi/hosts and put in the IPs and FQDNs, saved it, and restarted avahi with service avahi-daemon restart and it worked fine.
However whenever TrueNAS reboots, that information is cleared and no custom hosts remain in the above file.
I've Google searched, searched the forums here, tried editing the file both via the Shell interface and via SSH as root, but nothing sticks.

Should I be changing this somewhere else, or how can I ensure TrueNAS keeps the same avahi host table?

Thanks so much for any and all help regarding this!

 

[Samuel Tai]

Have each VM advertise its own Avahi hosts info. The TrueNAS middleware regenerates the Avahi hosts file on every boot, based on the Service Announcement settings under Network->Global Configuration.

 

[ITOperative]

Have each VM advertise its own Avahi hosts info. The TrueNAS middleware regenerates the Avahi hosts file on every boot, based on the Service Announcement settings under Network->Global Configuration.

Unfortunately ESXi does not have Avahi so that wasn't an option, hence why I was hoping to resort to using TrueNAS, as it has to boot agnostic of whether any other VM is scheduled to boot.

Also is there any way to keep it from regenerating the avahi hosts file?
"Host Name Database" doesn't appear to be associated with it, so I'd have no problem disabling that particular file from being regenerated if possible.
Thanks so much for the very prompt and informative reply!

 

[jgreco]

Also is there any way to keep it from regenerating the avahi hosts file?

You could hack up the middleware, but that's a tragically bad idea. It isn't meant to do this.

(and its storage supports other VMs on the parent host), I was hoping to use TrueNAS' instance of Avahi to do the advertising.

That's not a normal function of a hypervisor. Hypervisors run virtual machines. Your networking infrastructure and name resolution is a different problem entirely.

Why don't you just spin up a jail and then build the avahi port?

Around here, we build stuff from source from the ground up, and it looks like avahi is a bit crap-filled, but it looks like the worst of it is probably CUPS, which makes sense because that would generally be integrated.

Code:

total 49000
dr-xr-xr-x   6 root    wheel      1024 Sep  1  2021 .
dr-x--x--x  19 root    wheel       512 Sep  1  2021 ..
-rw-r--r--   1 root    wheel  25755357 Aug 30  2021 Python-3.9.7.tgz
-rw-r--r--   1 root    wheel       290 Sep  1  2021 Python-run
-rw-r--r--   1 root    wheel     10135 Sep  1  2021 airprint-generate.py
-rw-r--r--   1 root    wheel   1591458 Feb 18  2020 avahi-0.8.tar.gz
-rw-r--r--   1 nobody  wheel       771 Sep  1  2021 avahi-run
drwxr-xr-x   2 root    wheel       512 Sep  1  2021 cpan
-rw-r--r--   1 root    wheel     25930 Aug 31  2021 cpan-run
drwxr-xr-x   2 root    wheel       512 Sep  1  2021 cups
-rw-r--r--   1 root    wheel    987628 Jun 17  2021 gdbm-1.20.tar.gz
-rw-r--r--   1 root    wheel       251 Aug 31  2021 gdbm-run
-rw-r--r--   1 root    wheel   1957374 Aug  6  2021 get-pip.py
-rw-r--r--   1 root    wheel      3810 Apr 19  2020 installstuff
-rw-r--r--   1 root    wheel    340474 Oct 17  2009 libdaemon-0.14.tar.gz
-rw-r--r--   1 root    wheel       251 Aug 31  2021 libdaemon-run
-rw-r--r--   1 root    wheel   1082234 Aug  1  2019 libevent-2.1.11-stable.tar.gz
-rw-r--r--   1 root    wheel       251 Aug 31  2021 libevent-run
-rw-r--r--   1 root    wheel      9713 Sep  1  2021 makeitall
drwxr-xr-x   5 root    wheel       512 Sep  1  2021 old
-rw-r--r--   1 root    wheel  18038522 Jan 23  2021 perl-5.32.1.tar.gz
-rw-r--r--   1 root    wheel       429 Mar 30  2021 perl-run
drwxr-xr-x   2 root    wheel       512 Sep  1  2021 pythonstuff
-rw-r--r--   1 root    wheel      5496 Sep  1  2021 rc.avahi.src
-rw-r--r--   1 root    wheel        11 Aug 31  2021 solapps-avahi-supportedver
-rw-r--r--   1 root    wheel        23 Sep  1  2021 solapps-avahi-version

The list of dependencies is pretty minimal and I would think that this could be built in a jail pretty easily.

 

[ITOperative]

You could hack up the middleware, but that's a tragically bad idea. It isn't meant to do this.



That's not a normal function of a hypervisor. Hypervisors run virtual machines. Your networking infrastructure and name resolution is a different problem entirely.

Why don't you just spin up a jail and then build the avahi port?

Around here, we build stuff from source from the ground up, and it looks like avahi is a bit crap-filled, but it looks like the worst of it is probably CUPS, which makes sense because that would generally be integrated.

Code:

total 49000
dr-xr-xr-x   6 root    wheel      1024 Sep  1  2021 .
dr-x--x--x  19 root    wheel       512 Sep  1  2021 ..
-rw-r--r--   1 root    wheel  25755357 Aug 30  2021 Python-3.9.7.tgz
-rw-r--r--   1 root    wheel       290 Sep  1  2021 Python-run
-rw-r--r--   1 root    wheel     10135 Sep  1  2021 airprint-generate.py
-rw-r--r--   1 root    wheel   1591458 Feb 18  2020 avahi-0.8.tar.gz
-rw-r--r--   1 nobody  wheel       771 Sep  1  2021 avahi-run
drwxr-xr-x   2 root    wheel       512 Sep  1  2021 cpan
-rw-r--r--   1 root    wheel     25930 Aug 31  2021 cpan-run
drwxr-xr-x   2 root    wheel       512 Sep  1  2021 cups
-rw-r--r--   1 root    wheel    987628 Jun 17  2021 gdbm-1.20.tar.gz
-rw-r--r--   1 root    wheel       251 Aug 31  2021 gdbm-run
-rw-r--r--   1 root    wheel   1957374 Aug  6  2021 get-pip.py
-rw-r--r--   1 root    wheel      3810 Apr 19  2020 installstuff
-rw-r--r--   1 root    wheel    340474 Oct 17  2009 libdaemon-0.14.tar.gz
-rw-r--r--   1 root    wheel       251 Aug 31  2021 libdaemon-run
-rw-r--r--   1 root    wheel   1082234 Aug  1  2019 libevent-2.1.11-stable.tar.gz
-rw-r--r--   1 root    wheel       251 Aug 31  2021 libevent-run
-rw-r--r--   1 root    wheel      9713 Sep  1  2021 makeitall
drwxr-xr-x   5 root    wheel       512 Sep  1  2021 old
-rw-r--r--   1 root    wheel  18038522 Jan 23  2021 perl-5.32.1.tar.gz
-rw-r--r--   1 root    wheel       429 Mar 30  2021 perl-run
drwxr-xr-x   2 root    wheel       512 Sep  1  2021 pythonstuff
-rw-r--r--   1 root    wheel      5496 Sep  1  2021 rc.avahi.src
-rw-r--r--   1 root    wheel        11 Aug 31  2021 solapps-avahi-supportedver
-rw-r--r--   1 root    wheel        23 Sep  1  2021 solapps-avahi-version

The list of dependencies is pretty minimal and I would think that this could be built in a jail pretty easily.

I completely forgot about jails, that's a great idea!

I mainly wanted to have the primary VM handle it since the hypervisor couldn't (understandably), and I think a jail would do it nicely.
Thanks a bunch!

 

[jgreco]

I completely forgot about jails, that's a great idea!

I mainly wanted to have the primary VM handle it since the hypervisor couldn't (understandably), and I think a jail would do it nicely.
Thanks a bunch!

You could do a VM for Avahi as well, but the jail will be much more resource efficient if you can go that route. Would be pleased to hear how this turns out, it's an interesting edge case of network design; not many people seem to be all that focused on Avahi/DNS-SD.

 

[ITOperative]

Would be pleased to hear how this turns out, it's an interesting edge case of network design; not many people seem to be all that focused on Avahi/DNS-SD.

I didn't want to run and apply a dedicated DNS server for local services, and Avahi/mDNS seems to do the trick well enough, as well as being pretty easy to add to if/when needed. I'd probably move away from it if I every decided to make a full-blown DC and add a DNS server, but for now it suits my needs perfectly.


Also I decided to document each step of the way verbosely in case anyone else wants to do this and finds this in Google or something:

(As of writing, it doesn't work after reboot, but I feel I'm 90% of the way there.)

My first issue was networking.
The jail errored out saying that acquiring an IP from DHCP failed.
I first had to add a bridge interface, bridge0, as it wasn't automatically created.
After, I still had the issue, and eventually found this point on the TrueNAS forums in which someone also running inside ESXi had to modify their vSwitch. I had to simply change Promiscuous Mode from inherit to accept and it worked fine.
Stopped and re-started the Jail to ensure that worked, which it did.
Tested and confirmed networking was fine by using pkg install nano for my preferred text editor, then the same for pkg install avahi to install Avahi.

After Avahi was installed, I used nano /etc/rc.conf and added avahi_daemon_enable="YES" so it could be started with service avahi-daemon start
Once I confirmed the service started without issue, I used nano /usr/local/etc/avahi/hosts to edit the hosts file, and then added them similar to any hosts file, as IP FQDN like: 192.168.1.123 yourservice.local

After my changes were done, I ran service avahi-daemon restart to apply them, then I pinged one of the newly added entries to test and verify it resolved.
I used service avahi-daemon enable then restarted the jail and used service avahi-daemon status to confirm if the service started with the jail, which it did not; I researched further and eventually found that both of these need added to /etc/rc.conf for it to work:

Code:

avahi_daemon_enable="YES"
avahi_dnsconfd_enable="YES"

Restarting the jail once more finally confirmed the service started with the jail, however it was not providing changes to the mDNS.

I continued troubleshooting, trying to start or restart via crontab, setup netwait in case the service was starting before it got network, but nothing seems to get it to function until you log into the shell and restart it.
At the time of writing, I need to drop this for the night due to work in the morning, but so far that's pretty good progress.

A little extra-verbose, but I know everyone's level of knowledge with technology is different, so I hope I put this in a very useful, linear, and concise way.

 

[Patrick M. Hausen]

Unfortunately ESXi does not have Avahi so that wasn't an option, hence why I was hoping to resort to using TrueNAS, as it has to boot agnostic of whether any other VM is scheduled to boot.

He probably advised to run Avahi inside each individual VM. Works great, that's what I do. Avahi in default configuration in each jail and VM.

So when at home in my own network I can always use ssh <nodename>.local.

 

[ITOperative]

He probably advised to run Avahi inside each individual VM. Works great, that's what I do. Avahi in default configuration in each jail and VM.

So when at home in my own network I can always use ssh <nodename>.local.

Well he's right about the resource efficiency of using a jail, if possible:

You could do a VM for Avahi as well, but the jail will be much more resource efficient if you can go that route.

Running things in a single jail, or even VM, rather than one per each VM would be better resource-wise since you can advertise multiple hosts regardless of local or external to the machine running Avahi.
To top it off, since all VMs are reliant on TrueNAS to boot before they can, since TrueNAS is supplying an NFS share to its parent host, ESXi, it makes more sense to target TrueNAS as the highest-most available machine that can run it.

Currently my only problem is that Avahi-daemon isn't making the mDNS advertisements despite running when the jail starts, and I'm not sure where to go with that without very roughly troubleshooting it over wireshark and learning how to follow mDNS broadcasts.

 

[ITOperative]

Also I decided to document each step of the way verbosely in case anyone else wants to do this and finds this in Google or something:

(As of writing, it doesn't work after reboot, but I feel I'm 90% of the way there.)
...
Restarting the jail once more finally confirmed the service started with the jail, however it was not providing changes to the mDNS.
...
At the time of writing, I need to drop this for the night due to work in the morning, but so far that's pretty good progress.

Alright so after more testing than should've been needed, I figured it out, and I feel a bit like an idiot.
First off, fun fact: don't set up an mDNS advertisement pointing to a router/gateway!
I figured I'd use 192.168.1.1 router as a test, and then subsequent tests as 192.168.1.1 test1.local so as not to apply a bunch of advertisements floating across anything syncing with the mDNS. Turns out, mDNS doesn't play kindly to giving the router a hostname; not sure why, but I was only testing so I don't care lol

I also confirmed the two required /etc/rc.conf entries needed, and one of the ones I mentioned earlier was not it.


So just to summarize my experience, I'm going to list the steps in a cleaner order, for anyone else who may stumble upon this...
(Lord knows my google-fu tends to bring me to these forums a LOT, and they are incredibly helpful. As a community you all rock!)

Step 1 - Prep your jail...
Create the jail, ensure VNET is on (as I read you won't get broadcasts if VNET isn't on), and double-check that a bridge was created in Network > Interfaces on TrueNAS. If not, create one and ensure it has the same name as the bridge in Jails > (your jail) > Edit > Network Properties > Interfaces.

Step 2 - Download stuff...
Optional, add nano because it's a great and easy text editor (or an editor of your choice):
pkg install nano
Install Avahi-daemon:
pkg install avahi
Optional, but recommended by Avahi when I was debugging its output, install nss (look up nss-mdns):
pkg install nss

Step 3 - Enable everything in /etc/rc.conf...
Use nano /etc/rc.conf to open the file in nano, then add these lines:

Code:

dbus_enable="YES"
avahi_daemon_enable="YES"

(I believe dbus came with FreeBSD and/or Avahi, as I did not have to install it. But if things don't work, maybe try installing it.)

Step 4 - Configure your hosts...
This is the fun part...add your hosts to /usr/local/etc/avahi/hosts similar to how you would for most hosts files.
For example, if we wanted to give server.local to the IP address 192.168.1.123, and server2.local to 192.168.1.124, we would first open the file for editing:
nano /usr/local/etc/avahi/hosts
Then add the following lines to the bottom:

Code:

192.168.1.123 server.local
192.168.1.124 server2.local

Step 5 - Enable and run the service, then test...
Once you've saved your avahi hosts file, you want to enable the server so it runs on boot, start the service, and test.
Enable the service:
service avahi-daemon enable
Start the service:
service avahi-daemon start
Then ensure it's still running after:
service avahi-daemon status

After that, I'd recommend stopping the service, adding a test host to your avahi hosts file, and rebooting the jail.
If everything worked fine, the service should automatically start, and you should be able to use a different device to ping the new host you added.

Thanks everyone for your help, ideas, and advice!
I hope this helps anyone else with this issue. So far, it's been working great for me!