Issue with MAC addresses in jails

[maffo]

Hello all,
I am opening a new thread as I found a similar one that is about 3 years old and doesn't contain any useful information.

So here's a bit of background history: my FreeNAS box runs 3 different jails for 3 different services, I don't use static IPs but DHCP with reservations based on MAC addresses.
I was running 9.3 and had 3 jails running with the following MACs: 02:ff:b0:00:06:0b, 02:97:54:00:0c:0b, 02:ff:b0:00:08:0b. After replacing the boot drive and starting from scratch with a new 9.10 image, I re-created my jails and re-assigned them the old MAC address they used to have, to get back the same IP addresses from the DHCP (this operation has been done through the FreeNAS GUI).
This morning, I created a new jail and I have been extremely surprised in finding that it had been created with a duplicate MAC address of one that I already have!

Code:

[*****@babylon] /mnt/data/*****# jls
   JID  IP Address      Hostname                      Path
     1  -               mysql.lan                     /mnt/data/jails/mysql
     2  -               owncloud_1                    /mnt/data/jails/owncloud_1
     3  -               transmission.lan              /mnt/data/jails/transmission
     6  -               museek                        /mnt/data/jails/museek

[*****@babylon] /mnt/data/*****# foreach i (1 2 3 6)
foreach? jexec $i ifconfig -a | grep ether
foreach? end
        ether 02:ff:b0:00:06:0b
        ether 02:97:54:00:0c:0b
        ether 02:ff:b0:00:08:0b
        ether 02:ff:b0:00:08:0b

Jail 6 is obviously the one that was crated this morning.
In the messages file I can see the following entries (taken from the moment I create the jail in the GUI):

Code:

[*****@babylon] /mnt/data/*****# tail -n 41 /var/log/messages
Apr 16 08:31:20 babylon kernel: epair3a: link state changed to DOWN
Apr 16 08:31:20 babylon kernel: epair3a: link state changed to DOWN
Apr 16 08:31:20 babylon kernel: epair3b: link state changed to DOWN
Apr 16 08:31:20 babylon kernel: epair3b: link state changed to DOWN
Apr 16 08:31:21 babylon kernel: ifa_del_loopback_route: deletion failed: 48
Apr 16 08:31:21 babylon Freed UMA keg (udp_inpcb) was not empty (60 items).  Lost 6 pages of memory.
Apr 16 08:31:21 babylon Freed UMA keg (udpcb) was not empty (668 items).  Lost 4 pages of memory.
Apr 16 08:31:21 babylon Freed UMA keg (ripcb) was not empty (30 items).  Lost 3 pages of memory.
Apr 16 08:31:21 babylon hhook_vnet_uninit: hhook_head type=1, id=1 cleanup required
Apr 16 08:31:21 babylon hhook_vnet_uninit: hhook_head type=1, id=0 cleanup required
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs list -H -o name '/mnt/data/jails/.warden-template-standard'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs get -H origin '/mnt/data/jails/transmission'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs get -H origin '/mnt/data/jails/museek'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs get -H origin '/mnt/data/jails/mysql'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs get -H origin '/mnt/data/jails/owncloud_1'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs list -H -o name '/mnt/data/jails/.warden-template-standard'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs get -H origin '/mnt/data/jails/transmission'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs get -H origin '/mnt/data/jails/museek'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs get -H origin '/mnt/data/jails/mysql'
Apr 16 08:31:31 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/zfs get -H origin '/mnt/data/jails/owncloud_1'
Apr 16 08:31:56 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/ping -q -t 2 -o 192.168.1.1
Apr 16 08:31:58 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/ping -q -t 2 -o 192.168.1.2
Apr 16 08:31:58 babylon kernel: arp: 192.168.1.210 moved from 02:ff:60:00:05:0a to 38:ea:a7:ab:ed:32 on epair1b
Apr 16 08:32:16 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/ping -q -t 2 -o 192.168.1.1
Apr 16 08:32:17 babylon manage.py: [common.pipesubr:61] Popen()ing: /sbin/ping -q -t 2 -o 192.168.1.2
Apr 16 08:32:21 babylon warden: Building new Jail... Please wait...
Apr 16 08:32:21 babylon warden: zfs clone data/jails/.warden-template-standard@clean data/jails/museek
Apr 16 08:32:22 babylon warden: Success!
Apr 16 08:32:22 babylon warden: Jail created at /mnt/data/jails/museek
Apr 16 07:32:23 babylon devd: Executing '/etc/pccard_ether epair3a start'
Apr 16 07:32:23 babylon devd: Executing '/etc/pccard_ether epair3b start'
Apr 16 07:32:23 babylon devd: Executing '/etc/rc.d/dhclient quietstart epair3a'
Apr 16 08:32:23 babylon epair3a: Ethernet address: 02:ff:60:00:07:0a
Apr 16 08:32:23 babylon epair3b: Ethernet address: 02:ff:b0:00:08:0b
Apr 16 08:32:23 babylon kernel: epair3a: link state changed to UP
Apr 16 08:32:23 babylon kernel: epair3a: link state changed to UP
Apr 16 08:32:23 babylon kernel: epair3b: link state changed to UP
Apr 16 08:32:23 babylon kernel: epair3b: link state changed to UP
Apr 16 08:32:23 babylon kernel: epair3a: promiscuous mode enabled
Apr 16 08:32:23 babylon kernel: ng_ether_ifnet_arrival_event: can't re-name node epair3b
Apr 16 08:32:23 babylon kernel: ng_ether_ifnet_arrival_event: can't re-name node epair3b

My configuration is the following:
HP Microserver G7
AMD Turion 2 Neo
16 GB RAM
4x 3TB WD Red in RAIDZ1
FreeNAS 9.10 STABLE

I would treat this situation as a bug: the system should check which MAC addresses are already in use before generating a new one and assign it to a jail; I'm pretty confident the system keeps track of the MACs that are automatically generated and assigned, but this should be extended also to the ones that are decided "arbitrarily" by the user. By not doing so, we can end up with multiple shared MACs on the network (and in my specific case also duplicate IPs) which can cause networking issues and lead to disruption. I am not sure whether this issue can happen also when creating network interfaces and/or link aggregations as I don't use those features yet.

If anybody could help and at least confirm it's not me who is doing something wrong, I'd really appreciate that.

Regards

 

[dlavigne]

Please create a bug report at bugs.freenas.org and post the issue number here.

 

[maffo]

Opened bug 14746 as requested.

 

[SigLinJo]

Hi. I just wanted to add that I experienced a similar problem. I created two new jails in 9.10 and migrated my proxy and my owncloud servers to them and nothing worked. After several hours of beating my head against it I noticed that the two AUTOMATICALLY assigned MAC-addresses where identically when my router complained that two clients connected through the same instance and it shifted back and forth between the to ip addresses of the jails. I fixed it by manually change one of the MACs through the webgui. Thought it would be interesting to share that it can happen with automatically assigned MACs also.

 

[maffo]

The BUG has been closed already with the following note:
MAC assignment is basically a boolean: Either you let it auto-assign all the addresses, or you assign them all manually. The jail code could ostensibly be enhanced to not be this way, but as it's essentially EOL'd in the 9.x branch it's not something on our roadmap. Pull requests on github from the community will be reviewed.

I didn't try it in the 10.x release yet as it's not stable apparently.

 

[SigLinJo]

Yea I read that to. Just thought that it might be interesting for someone to know that it happened even with only automatically assigned MACs. Thank you for replying!