Bind Interface Bridge to NFS

[jsmith9255]

I created a bridge between two 10gbe interfaces on the same NIC to service 2 esx hosts (one directly attached to each interface) in a switchless 10gbe network. The bridge works as expected and was configured in /conf/base/etc/rc.conf. Both esx hosts can ping one another and the freenas box. However, I am having difficulty adding the bridge to the list of Freenas interfaces as it does not show up and thus cannot bind it to NFS.

The end goal would be to bind the bridge interface to NFS so that I can share a storage pool between the two esx hosts and use the 10gbe connection for vmotion. This has been successfully set up here, but I cannot for the life of me figure out how. Any advice/guidance would be greatly appreciated.

Thanks!

 

[sretalla]

You're going to need to give us at least ifconfig so we can make some kind of attempt to help.

When I'm looking at the NFS config (on 11.2-U3), I see the option to bind by IP address, not interface. I see the possibility to bind to both of the IP addresses my box has, either or none (presumably 0.0.0.0 = all)... I don't see why you need to select the bridge.

 

[jsmith9255]

You're going to need to give us at least ifconfig so we can make some kind of attempt to help.

Code:

root@freenas[~]# ifconfig -a
ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 900                             0
        options=e403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS                             UM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether b8:ca:3a:66:84:98
        hwaddr b8:ca:3a:66:84:98
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
        status: active
ix1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 900                             0
        options=e403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS                             UM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether b8:ca:3a:66:84:9a
        hwaddr b8:ca:3a:66:84:9a
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
        status: active
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS                             UM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether b8:ca:3a:66:84:9c
        hwaddr b8:ca:3a:66:84:9c
        inet 192.168.10.17 netmask 0xffffff00 broadcast 192.168.10.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS                             UM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether b8:ca:3a:66:84:9d
        hwaddr b8:ca:3a:66:84:9d
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        ether 02:51:aa:30:fc:00
        inet 10.0.10.1 netmask 0xffffff00 broadcast 10.0.10.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        groups: bridge
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: ix1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 2 priority 128 path cost 2000
        member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 2000

Ifconfig as requested. Additionally, here is the bridge config in rc.conf:

Code:

# Auto enable Bridge
gateway_enable="YES"
cloned_interfaces="bridge0"
autobridge_interfaces="bridge0"
autobridge_bridge0="ix0 ix1"
ifconfig_ix0="up"
ifconfig_ix1="up"
ifconfig_ix0="mtu 9000"
ifconfig_ix1="mtu 9000"
ifconfig_bridge0="inet 10.0.10.1 netmask 255.255.255.0 mtu 9000"

When I'm looking at the NFS config (on 11.2-U3), I see the option to bind by IP address, not interface. I see the possibility to bind to both of the IP addresses my box has, either or none (presumably 0.0.0.0 = all)... I don't see why you need to select the bridge.

I am also running 11.2-U3, but I don't see how I would input 0.0.0.0, as the only option is the one active interface I have on my box. There does not seem to be an option to input an IP address but perhaps I just missed it. See photo:

Thanks for your help.

 

[sretalla]

You can un-tick the one item in the list, meaning no binding (the setting I have on both IP addresses, the tick turns into a dash), which results in the service binding to all IPs.

If your bridge doesn't have an IP, then no binding should need to happen.

 

[sretalla]

Additionally, here is the bridge config in rc.conf:

I almost forgot... if this is something you have edited manually in the file, you're going to get disappointed somewhere down the line when FreeNAS nails the file (it's automatically erased and re-built when the system sees the need to change the file and also just for no good reason sometimes).

As I understand it, all of those can be made into tunables of type rc.conf. YOu should just be able to take the = out and put variables (left hand of the equation) and values (right hand) in accordingly. This way, you will survive upgrades and reboots of FreeNAS.

 

[jsmith9255]

You can un-tick the one item in the list, meaning no binding (the setting I have on both IP addresses, the tick turns into a dash), which results in the service binding to all IPs.

Done. However, there are still connectivity problems. I have been doing some additional testing and think it might be an issue with my bridge. I am able to mount all shares using the IP address of my management port (igb0 : 192.168.10.17) as the NFS target successfully. However, no dice over my 10gbe network (10.0.10.0/24). So it does not appear to be an issue with the way the shares are set up on FreeNAS's side. I took one of my virtual machines and added it to the vmotion/nfs portgroup(the 10gbe network), assigned it an ip of 10.0.10.4 and was able to successfully ping everything on the 10gbe network. I then tried to mount an nfs share using the 10gbe bridge interface (10.0.10.1) as the target but it timed out with the error:

Code:

[root@localhost ~]# mount -t nfs 10.0.10.1:/mnt/datastore /mnt/
mount.nfs: Connection refused

I thought it might be an issue where mountd or nfs was not running on the bridge interface, but rpcinfo says everything is good (i think):

Code:

[root@localhost ~]# rpcinfo -p 10.0.10.1
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100000    4     7    111  portmapper
    100000    3     7    111  portmapper
    100000    2     7    111  portmapper
    100005    1   udp    703  mountd
    100005    3   udp    703  mountd
    100005    1   tcp    703  mountd
    100005    3   tcp    703  mountd
    100003    2   tcp   2049  nfs
    100003    3   tcp   2049  nfs
    100024    1   udp    854  status
    100024    1   tcp    854  status
    100021    0   udp    609  nlockmgr
    100021    0   tcp   1004  nlockmgr
    100021    1   udp    609  nlockmgr
    100021    1   tcp   1004  nlockmgr
    100021    3   udp    609  nlockmgr
    100021    3   tcp   1004  nlockmgr
    100021    4   udp    609  nlockmgr
    100021    4   tcp   1004  nlockmgr

As far as I know I don't have any firewalls set up, at least I did not enable anything, and as I understand it FreeNAS does not have any by default. I also could not find anything in the freeBSD documentation on bridging to make me think that there would be some type of connection blocking. So this is a bit of a head-scratchier. If there is more comprehensive testing that I could be doing to further narrow down the problem, please let me know.

Thanks!

 

[jsmith9255]

This issue has been resolved. It looks like there was a temporary issue with Mountd that kept it from properly serving the NFS share over 10.0.10.1. A reboot has fixed the problem.

Thanks for your help.