VM's in bhyve getting same IP's

[rfanch3r]

I am getting the same ip being assigned to 2 different VM's, is there some trick to getting different IP's that I am not aware of?

1589189114 igb0 0 00:a0:98:58:c6:fc 192.168.1.108 ARP_REQ
1589189146 igb0 0 00:a0:98:0e:3d:52 192.168.1.108 ARP_REQ

I spent a good portion of the evening thinking is was my router but I switched the dhcp server to something else and its doing the exact same thing. I thought changing the mac address would get you a new dhcp IP but when I change it, I get the same one.

The only thing I did the same was to make a new VM, configure it and clone it so I could cut down on the install time. When cloning the new VM I made sure to generate a new mac address. Both are using VirtIO but changing to Intel doesn't seem to change anything, also long term the VM stops responding if left on Intel.

VM1:
lspci

00:00.0 Host bridge: Network Appliance Corporation Device 1275
00:03.0 SATA controller: Intel Corporation 82801HR/HO/HH (ICH8R/DO/DH) 6 port SATA Controller [AHCI mode]
00:04.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:1d.0 VGA compatible controller: Device fb5d:40fb
00:1e.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller
00:1f.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]

ifconfig
enp0s4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.108 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::2a0:98ff:fe58:c6fc prefixlen 64 scopeid 0x20<link>
ether 00:a0:98:58:c6:fc txqueuelen 1000 (Ethernet)
RX packets 16621 bytes 3273153 (3.2 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 96 bytes 11307 (11.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

VM2:
lspci

00:00.0 Host bridge: Network Appliance Corporation Device 1275
00:03.0 SATA controller: Intel Corporation 82801HR/HO/HH (ICH8R/DO/DH) 6 port SATA Controller [AHCI mode]
00:04.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:1d.0 VGA compatible controller: Device fb5d:40fb
00:1e.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller
00:1f.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]

ifconfig
enp0s4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.108 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::2a0:98ff:fe0e:3d52 prefixlen 64 scopeid 0x20<link>
ether 00:a0:98:0e:3d:52 txqueuelen 1000 (Ethernet)
RX packets 1442 bytes 280464 (280.4 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 92 bytes 12416 (12.4 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

 

[Samuel Tai]

Try completely deleting the NIC on one, and then adding a completely new NIC.

 

[rfanch3r]

I removed it like you suggested, getting the same IP on both.

 

[Samuel Tai]

OK, then what's likely happening is both VMs are configured with the same DHCP client ID, and this is being used instead of the MAC address. What guest OSs do you have on each VM?

 

[Samuel Tai]

According to https://superuser.com/questions/1321328/can-we-change-the-dhcp-client-id-from-etc-network-interfaces, this can happen on Ubuntu if the hostnames of both VMs are the same.

 

[rfanch3r]

Each guest is Ubuntu 18.04 and I went ahead and ran the following and it didn't change anything:
hostnamectl set-hostname (something different), I verified that the name changed and rebooted, still same ip
Then I went in and changed the /etc/hosts file and made that change, still same ip
I do see that send dhcp-client-identifier option but in both VM's they are not active

I did take a peek in /etc/netplan but there really isnt anything in there that would send the exact identifier and ask for the exact same ip.

 

[Samuel Tai]

Here's a suggestion to switch netplan back to using MACs: https://superuser.com/questions/1338510/wrong-ip-address-from-dhcp-client-on-ubuntu-18-04.

 

[rfanch3r]

wow, I would have never thought to look there, adding that dhcp-identifier: mac to netplan worked. I guess everything I read about why netplan is crap was true. What a pain. Thank you!

Your search kung-fu is way way better than mine, I was searching for "2 VM's getting same ip" and other variations on that, what was it you searched for to find this?

 

[Samuel Tai]

I searched for "ubuntu dhcp client id".

 

[rfanch3r]

Do you happen to know how to change the VM hostname without having to log into each vm? Something automated or is this all still a manual process?

 

[Samuel Tai]

Unfortunately, bhyve doesn't have the orchestration hooks to accomplish this. You'll have to login to each guest in turn.

 

[rfanch3r]

Bummers, ok thanks.

 

[Samuel Tai]

OK, I did some more reading on Ubuntu, it appears the proper way to do this, if you want to keep netplan's default behavior and not override it with the MAC address, is to delete /etc/machine-id, and then generate a new one using systemd-machine-id-setup --commit. Afterwards, reboot, and a different IP should be assigned from DHCP then.

 

[rfanch3r]

I will add that to my notes however, since I am making a master vm to clone, adding the dhcp-identifier: mac to the master VM lets me reduce what I need to remember. As of now all I need to do is change the hostname and hosts file.

I do thank you though for the help, I would have never gotten to the correct conclusion had you not been so patient.

 

[Patrick M. Hausen]

If you change the hostname after installation, make sure to check /etc/cloud/cloud.cfg if it exists. When present, cloud-init will by default reset your hostname on every reboot. You need to make sure this line is as shown:

Code:

preserve_hostname: true

 

[rfanch3r]

Ohh nice! Didn't even know that setting was present.