Steve@xxx.xxx.xxx.xxx
Dabbler
- Joined
- Aug 5, 2013
- Messages
- 23
FreeNAS-8.3.1-RELEASE-p2-x64
HP DL380 G5 Dual Xeon Quad Core 5450 3GHz 8GB ECC RAM, E200 RAID Controller, 3-146GB 10k SAS in RAID1 with spare, 2 NC373i Multifunction Gigabit Network Adapters with TCP/IP Offload Engine
On initial setup w/DHCP I get interfaces bce0 on 192.168.1.92 and bce1 on 192.168.1.93 and can ping both and get into GUI from a laptop on same subnet 192.168.1.0/24. The goal is a small iSCSI SAN as part of an Oracle VM virtualization project. I have been able to successfully set that up and see LUNS from the Oracle side with MPIO, but now I want to use different subnets to separate the iSCSI traffic from other network traffic with VLANS.
I'm using a Cisco 3550 L3 Switch with IP routing enabled. Switch Virtual Interfaces (SVIs) IPs are setup as 192.168.1.78/24, 10.0.10.254/24, 10.0.20.254/24, 10.0.30.254/24, 10.0.40.254/24, 10.0.50.254/24 and 10.0.60.254/24. All switchports are setup as trunks and I get the same results with or without VLANs configured in FreeNAS.
I've added IP address 10.0.50.1 to bce0 and 10.0.60.1 to bce1 which I want to use for the MPIO, so now the network summary looks like this in FreeNAS
Name IPv4 Address
bce0 10.0.50.1/24
bce0 192.168.1.92/24
bce1 192.168.2.93/24 DHCP had this at 192.168.1.93 but GUI forced change when adding IF IP 10.0.60.1
bce1 10.0.60.1/24
Nameserver
192.168.1.254
Default route
192.168.1.78
I can ping from the CLI on FreeNAS and the switch, to all hosts on all networks except for the two I need, and from the laptop to all hosts on all networks except the two FreeNAS MPIO IPs that I need to reach at 10.0.50.1 and 10.0.60.1. If I change bce0 from 10.0.50.1 to 10.0.40.1, then I can ping 10.0.50.254 from FreeNAS but not the 10.0.40.254 with console message of
ping: send to: Host is down
Looking at Packet traces with Wireshark on the laptop, when trying to ping 10.0.50.1 or 10.0.60.1, I can see the 3550 broadcasting ARP requests "who has 10.0.50.1?, Please tell 10.0.50.254" but there is no ARP reply from the FreeNAS interface at 10.0.50.1 and that seems to be the problem.
The netstat -rn output looks like this
Internet: Destination Gateway Flags Netif
10.0.50.0/24 link#1 U 0 4 bce0
10.0.50.1 link#1 UHS 0 0 lo0
10.0.60.0/24 link#2 U 0 0 bce1
10.0.60.1 link#2 UHS 0 0 lo0
38.229.71.1
192.168.1.254 UGHD3 0 16 bce0 3504
50.116.27.42
192.168.1.254 UGHD3 0 16 bce0 3504
127.0.0.1 link#9 UH 0 14159 lo0
192.168.1.0/24 link#1 U 0 3215 bce0
192.168.1.92 link#1 UHS 0 0 lo0
192.168.2.0/24 link#2 U 0 0 bce1
192.168.2.93 link#2 UHS 0 0 lo0
198.60.22.240 192.168.1.254 UGHD3 0 16 bce0 3504
Internet6: Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#9 UH lo0
::ffff:0.0.0.0/96
::1 UGRS lo0
fe80::%lo0/64 link#9 U lo0
fe80::1%lo0 link#9 UHS lo0
ff01::%lo0/32 fe80::1%lo0 U lo0
ff02::%lo0/32 fe80::1%lo0 U lo0
Thanks in advance for your help.
HP DL380 G5 Dual Xeon Quad Core 5450 3GHz 8GB ECC RAM, E200 RAID Controller, 3-146GB 10k SAS in RAID1 with spare, 2 NC373i Multifunction Gigabit Network Adapters with TCP/IP Offload Engine
On initial setup w/DHCP I get interfaces bce0 on 192.168.1.92 and bce1 on 192.168.1.93 and can ping both and get into GUI from a laptop on same subnet 192.168.1.0/24. The goal is a small iSCSI SAN as part of an Oracle VM virtualization project. I have been able to successfully set that up and see LUNS from the Oracle side with MPIO, but now I want to use different subnets to separate the iSCSI traffic from other network traffic with VLANS.
I'm using a Cisco 3550 L3 Switch with IP routing enabled. Switch Virtual Interfaces (SVIs) IPs are setup as 192.168.1.78/24, 10.0.10.254/24, 10.0.20.254/24, 10.0.30.254/24, 10.0.40.254/24, 10.0.50.254/24 and 10.0.60.254/24. All switchports are setup as trunks and I get the same results with or without VLANs configured in FreeNAS.
I've added IP address 10.0.50.1 to bce0 and 10.0.60.1 to bce1 which I want to use for the MPIO, so now the network summary looks like this in FreeNAS
Name IPv4 Address
bce0 10.0.50.1/24
bce0 192.168.1.92/24
bce1 192.168.2.93/24 DHCP had this at 192.168.1.93 but GUI forced change when adding IF IP 10.0.60.1
bce1 10.0.60.1/24
Nameserver
192.168.1.254
Default route
192.168.1.78
I can ping from the CLI on FreeNAS and the switch, to all hosts on all networks except for the two I need, and from the laptop to all hosts on all networks except the two FreeNAS MPIO IPs that I need to reach at 10.0.50.1 and 10.0.60.1. If I change bce0 from 10.0.50.1 to 10.0.40.1, then I can ping 10.0.50.254 from FreeNAS but not the 10.0.40.254 with console message of
ping: send to: Host is down
Looking at Packet traces with Wireshark on the laptop, when trying to ping 10.0.50.1 or 10.0.60.1, I can see the 3550 broadcasting ARP requests "who has 10.0.50.1?, Please tell 10.0.50.254" but there is no ARP reply from the FreeNAS interface at 10.0.50.1 and that seems to be the problem.
The netstat -rn output looks like this
Internet: Destination Gateway Flags Netif
10.0.50.0/24 link#1 U 0 4 bce0
10.0.50.1 link#1 UHS 0 0 lo0
10.0.60.0/24 link#2 U 0 0 bce1
10.0.60.1 link#2 UHS 0 0 lo0
38.229.71.1
192.168.1.254 UGHD3 0 16 bce0 3504
50.116.27.42
192.168.1.254 UGHD3 0 16 bce0 3504
127.0.0.1 link#9 UH 0 14159 lo0
192.168.1.0/24 link#1 U 0 3215 bce0
192.168.1.92 link#1 UHS 0 0 lo0
192.168.2.0/24 link#2 U 0 0 bce1
192.168.2.93 link#2 UHS 0 0 lo0
198.60.22.240 192.168.1.254 UGHD3 0 16 bce0 3504
Internet6: Destination Gateway Flags Netif Expire
::/96 ::1 UGRS lo0
::1 link#9 UH lo0
::ffff:0.0.0.0/96
::1 UGRS lo0
fe80::%lo0/64 link#9 U lo0
fe80::1%lo0 link#9 UHS lo0
ff01::%lo0/32 fe80::1%lo0 U lo0
ff02::%lo0/32 fe80::1%lo0 U lo0
Thanks in advance for your help.