SOLVED FreeBSD 11 CARP help

[kdragon75]

Hello, I'm trying to get ctl and CARP working to implement a shared storages HA configuration. I have the ctl HA setup* for the failover of the CTL front end ports. The part I'm hung up on is the CARP setup. It seems like it should be quite simple but both hosts ctlr-a and ctlr-b are stuck in
"Backup". I could use a second set of eyes to look over my config. Note, I am not using preempt as I want to implement hold down timers and flapper limits to prevent an endless loop of flip flopping.

ctlr-a:

Code:

root@ctlr-a:~ # ifconfig vmx0
vmx0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=60039b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
		ether 00:50:56:88:89:ac
		hwaddr 00:50:56:88:89:ac
		inet 192.168.1.243 netmask 0xffffff00 broadcast 192.168.1.255
		inet 192.168.1.245 netmask 0xffffffff broadcast 192.168.1.245 vhid 1
		nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
		media: Ethernet autoselect
		status: active
		carp: BACKUP vhid 1 advbase 1 advskew 1
root@ctlr-a:~ # cat /etc/rc.conf
hostname="ctlr-a"
defaultrouter="192.168.1.1"
ifconfig_vmx0="inet 192.168.1.243/24"
ifconfig_vmx0_alias0="inet vhid 1 pass pa$$w0rd advskew 1 alias
192.168.1.245/24"

sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"
zfs_enable="YES"
ctld_enable="yes"

ctlr-b:

Code:

root@ctlr-b:~ # ifconfig vmx0
vmx0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=60039b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
		ether 00:50:56:88:63:2d
		hwaddr 00:50:56:88:63:2d
		inet 192.168.1.244 netmask 0xffffff00 broadcast 192.168.1.255
		inet 192.168.1.245 netmask 0xffffffff broadcast 192.168.1.245 vhid 1
		nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
		media: Ethernet autoselect
		status: active
		carp: BACKUP vhid 1 advbase 1 advskew 2
root@ctlr-b:~ # cat /etc/rc.conf
hostname="ctlr-b"
defaultrouter="192.168.1.1"
ifconfig_vmx0="inet 192.168.1.244/24"
ifconfig_vmx0_alias0="inet vhid 1 pass pa$$w0rd advskew 2 alias
192.168.1.245/24"

sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"
zfs_enable="YES"
ctld_enable="yes"

disabling either hosts network via cable pull or ifconfig vmx0 down seems to have no effect and running tcpdump -T carp | grep carp shows nothing. Both hosts can ping each other and the gateway. As this is running in ESXi 6.7 I have also set the following policies on the port group.

Code:

Security
Promiscuous mode Accept
MAC address changes Accept
Forged transmits Accept

Any insights would be appreciated!

 

[kdragon75]

Pulled from FreeBSD forums

If you have multiple physical ports on the same vswitch, see item 4 on this article.

You were right on the money! Its now passing the IP back and forth as expected. I don't think i would have found this in a reasonable amount of time on my own!
Sidenote: Setting Net.ReversePathFwdCheckPromisc = 1 on the ESXi host works on the fly, no reboot required as a few have suggested.

 

[HoneyBadger]

Curious, are you using HAST devices for the pool vdevs, a dual-ported SAS enclosure, or just shared VMDKs in VMware?

 

[kdragon75]

Curious, are you using HAST devices for the pool vdevs, a dual-ported SAS enclosure, or just shared VMDKs in VMware?

At this point I'm working with a shared VMDK. This is more of a learning exercise than anything else. Eventually I would like to try this with a shared SAS enclosure and even with HAST. I need to find some white papers on both ctl HA and HAST to fully grasp what's going on and the limitations. I'm having a hard time finding good in depth technical information on either.