Dedicated iSCSI network interface with vmware

[AVSION]

Hi,

i have two servers freenas (supermicro) and vmware (dell) each mobo has 2 G nics. on the freenas i would like to use one nic for the freenas OS/management/UI and the second dedicated for iSCSI.
and on the vmware do the same one nic dedicated to the OS and the other to iSCSI. at this point i manage to configure the iSCSI on both servers and its working but using the same network 192.168.1.X on both the freenas and vmware.
when trying to add the second nic on the freenas it only allow me to be on different network 192.168.0.X. how do i configure the interfaces and the portal on the freenas but still using one network 192.168.1.X? on the vmware
i already configure/split the two interfaces using the vswitch and VMkernel.

BTW i'm using FN 11.2 BETA2 it took me hours to set the iSCASI, i end up finding that the new IU has bugs ( missing settings, port 3260 was closed when checking with nmap..etc). to fix it i removed all iSCSI configuration from freenas and vmware, then start over from the legacy UI and bang iSCSI is back. (not sure if i need to report this bugs)

i use vmware ESXi 6.7

let me know if you need more information

thank you in advance

 

[kdragon75]

when trying to add the second nic on the freenas it only allow me to be on different network 192.168.0.X.

This is because FreeNAS atemptemts to strictly follow the one interface per subnet per host rule and is the correct way of doing it.

how do i configure the interfaces and the portal on the freenas but still using one network 192.168.1.X? on the vmware
i already configure/split the two interfaces using the vswitch and VMkernel.

You need to clarify what your doing. Perhaps a screenshot of the vSwitch will help.

i end up finding that the new IU has bugs

You should expect lots of broken things and bug. Its BETA.

In short, you will have two subnets. Ideally each subnet would be in its own VLAN but for a simple home lab this is not required. On FreeNAS this should be self explanatory. On note on safety, you should limit the intators to a list of IPs or at the least, your storage network. In my example you would enter 192.168.70.0/24 for your allowed subnet.

On the ESXi side of things, you need to add a vmkernel port and assign an IP in the same subnet as the FreeNAS IP Storage interface. But before you do that, you have a few choices to make. The easiest is to add a second vSwitch and use your second NIC as the uplink. The advantage is that this is a clean and easy to understand way of doing this. The disadvantage is that its not flexible or redundant. I would prefer to use a single vswitch and use portgroup overrides. This allows us to have all port groups except the IP storage to use NIC1 unless the link goes down, then I can fail over to NIC2 and vice versa.

 

[AVSION]

Thank you for your reply very interesting. Sorry i might not supply all the information in my OP, in general what i'm trying to do is strictly for max performance, stability and relability reasons only (vm and storage), in a home lab learning environment so security is not an issue at this point unless i should consider it for other reasons. as well the storage should be available to all devices/hosts in one subnet/network 192.168.1.X/24 unless there is another simple way to do it without VLANs. One note about the setup, my freenas and vmware servers are in separate locations (same LAN) with one CAT6 point connected available as the dell (vmware) is bit noisy. considering the above goals and setup, what configuration on the freenas side would you recommend that will allow me to improve performance in my current setup?

On the ESXi side, yes i added a vmkernel port and assign an IP in the same subnet as the FreeNAS IP Storage interface. for the switches i created a second vSwitches for the iSCSI storage and used the second NIC as the uplink meaning 192.168.1.X/24 for the management and 192.168.2.X/24 for the iSCSI. this configuration works when using 2 separate subnets, connecting the second interface on the router and switch to .2.X/24 and moving the vmware server to the same location which i don't want to do as have many downsides.

I basically need is to decide, with your help and expertise, first the freenas configuration and then have a better idea of the ESXi end which sound like using a single vswitch and use portgroup overrides.

here is all the screenshots, let me know if you need the freenas screenshots as well. there is two things i need to look at later the virtual switch link redundancy which will be solved as you suggested with single vSwitch and in storage devices screenshot normal degraded alert.

Thank you

 

[kdragon75]

Yeah, that looks good. The "Normal, Degraded" is because you only have one path to the storage. Just use extreme caution with that cable or you will crash all of your VMs! Many people will get two dual port 10gbe cars and do the same thing point to point - no switch. It's the same setup doubled. In that case you might just add a 192.168.3.0/24 network.

 

[jgreco]

There's a resource available that discusses this.

https://forums.freenas.org/index.php?resources/multiple-network-interfaces-on-a-single-subnet.45/

 

[AVSION]

thank you guys, reading the resource lead me to Stux AIO build which is better way to go as i have all the resources in my current machine. already started having some issues with the AHCI but this is for another thread.

 

[RegularJoe]

you will want to use a cisco switch and trunk multiple VLANs on your two interfaces, do SRC-DST-IP hash. VMware will do it, if you have the free version you cannot use LACP but must use mode on for your port channel.