Main Network / iSCSI Network - Cannot get iSCSI to work at all

[kjparenteau]

I am not sure how to resolve this issue... I did a clean install of FreeNAS 11.0-U2 and also started with 6x New 3TB drives. No issues with the SMB Share service, or LAN traffic, etc. But when I try to add iSCSI in the mix, I cannot get it to work. I am trying to add my ESXi host and having major issues and it looks like FreeNAS is where the issue is. I will summarize...

Note: I am using Intel Quad Port VT/1000 adapters in both boxes.

ESXi host: 4 NIC's (2 for LAN, and 2 for iSCSI)
- LAN vSwitch (NIC1, NIC2) - 24 port switch (10.0.0.6/24) - Default VLAN
- iSCSI vSwitch (NIC3, NIC4) - 8 port switch (10.10.10.6/24) - VLAN10
- Both sets of NIC's are teamed on the switch and showing active
- From SSH, I can ping both the gateway address as well as other IP's on both networks
- Since I can ping addresses on both networks, I don't believe my issue is with ESXi.

FreeNAS: 4 NIC's (2 for LAN, and 2 for iSCSI)
- LAN vSwitch (NIC1, NIC2) - 24 port switch (10.0.0.5/24) - Default VLAN
- iSCSI vSwitch (NIC3, NIC4) - 8 port switch (10.10.10.5/24) - VLAN10
- Both sets of NIC's are setup in a LAGG presently as "Failover" just to keep things simple for now. I will worry about performance/LACP later.
- From SSH, I can ping the LAN Gateway (10.0.0.1/24), but not ANY address on 10.10.10.0/24.
- I have also broken the LAGG and tried NIC's by themselves. Same issue, I cannot talk to the 10.10.10.0/24 network.

I have tried many configurations with the same or worse results. I have the Global IPv4 Gateway as 10.0.0.1 (since that is where internet is), but to test I have removed it, I have tried setting static routes, and I have messed with VLAN configurations on the switch and on both the ESXi host and FreeNAS. Between all of these actions I am restarting the network adapter to ensure the settings are applied as it starts up, but still nothing. I am confused because this EXACT configuration used to work just fine on a previous setup. Has something changed from v9.X to v11 to break this? Please advise.

Thank you,
Kevin

 

[kjparenteau]

Nobody has any suggestions? I have been searching the forums and other sources for answers, I have also eliminated the main network out of the equation by putting in a standalone gigabit switch (which used to be the main iSCSI switch) that has no connection to the main network at all and that still didn't work. My 2 ESXi hosts can talk to each other over the main Management network as well as the iSCSI network, but FreeNAS is refusing to play ball. I could really use some help with this. I am not sure what to do at this point.

Thank you,
Kevin

 

[kjparenteau]

Also, it is worth mentioning, I have created appropriate portal on the iSCSI network, no auth required, initiators is set for ALL/ALL, added a target with no auth required, iSCSI file extent, and associated the target with the extent. Started service, and it starts just fine. Doesn't matter if the 2 ports for FreeNAS iSCSI are plugged into the main switch I want them in with a VLAN, or a dedicated switch that has ONLY iSCSI devices connected, the hosts cannot ping FreeNAS's iSCSI address. The hosts can ping each other, but they cannot ping FreeNAS at all. And of course FreeNAS cannot ping the host's iSCSI adapters. Please help! I am stuck without a working environment. :(

 

[kjparenteau]

I would really appreciate a little help with this... I'm at a standstill and not a single response yet... :(

 

[wblock]

From SSH, I can ping the LAN Gateway (10.0.0.1/24), but not ANY address on 10.10.10.0/24.
- I have also broken the LAGG and tried NIC's by themselves. Same issue, I cannot talk to the 10.10.10.0/24 network.

Doesn't matter if the 2 ports for FreeNAS iSCSI are plugged into the main switch I want them in with a VLAN, or a dedicated switch that has ONLY iSCSI devices connected, the hosts cannot ping FreeNAS's iSCSI address. The hosts can ping each other, but they cannot ping FreeNAS at all. And of course FreeNAS cannot ping the host's iSCSI adapters.

Both of these suggest routing issues. Or possibly firewall issues. I know that some things have changed due to the underlying operating system being updated. For instance, MTU on LAGGs has changed. The important differences should all be specified in the What's New section at the start of the user guide. I can't say whether any of those changes are at fault here, though.

 

[kjparenteau]

First off, thank you for responding! Not sure why it's taken this long for any response from anyone, but again, thank you.

So here's the thing... Isolating the adapters used for iSCSI from the ESXi hosts and the FreeNAS box to their own dedicated switch removes the routing piece out of the equation. Even simplifying further, going single adapter between the boxes to remove LAGG from the equation as well. ESXi Host 1 (10.10.10.6) can ping ESXi Host 2 (10.10.10.7), but neither of them can ping FreeNAS (10.10.10.5). And those are the only 3 adapters on that switch, and the switch has no uplink to any other device. It is 100% standalone.

There is clearly a piece missing with FreeNAS that I am missing, or the functionality is broken. I had this working just fine with 9.3, rinse and repeat with 11 and it doesn't work at all... I am very confused by this because I have run a native VLAN along side a dedicated iSCSI switch for YEARS and why I am having an issue now especially after setting this up multiple times over the years is very frustrating.

 

[wblock]

For full detail, which type and brand of network adapters are these?

 

[kjparenteau]

I outlined this in the first post, but each box is using the exact same adapters, pulled from working Dell Servers, and also tested separately outside of FreeNAS to verify it was working. Note, this same adapter was working just fine with this exact configuration in FreeNAS 9.X. My only issue seems to be related with moving to v11...

NIC: Intel Pro 1000 VT Quad Port PCIe Gigabit Ethernet Network Adapter

 

[wblock]

Sorry, I was looking in the list and not before it.

 

[kjparenteau]

No worries at all, I just appreciate the assistance and any guidance you can help provide me.

 

[wblock]

One thing to check: are you setting a non-default MTU for any of these interfaces?

 

[kjparenteau]

No changes to MTU as of yet. I assumed since defaults worked fine previously, that it would work this time around as well.

 

[wblock]

That would be my assumption also. One thing that changed in FreeBSD 11 was that MTU could no longer be set on LAGG devices, just on the individual network interfaces. That could be the same for other properties. But if you don't have anything like that set, it doesn't matter.