iSCSI with VMware vSphere 6.7 - how to enable multipath in vSphere to TrueNAS 13 U1

[Murphy1138]

Hello, I have searched the internet and am coming up blank and have hit a wall and for the life of me I can't figure out why I can't get vSphere to multipath to TrueNAS 13 over iSCSI.
Any advice would be great.

my environment
3x ESXi hosts, with 2x dedicated iSCSI NICs going to iSCSI A and iSCSI B on my storage switch.
1x QNAP SAN 2x iSCSI NICs going to A and B (multi-path works fine here)
2x IBM SANS 4x iSCSI NICs going to A and B (multi-path works fine here too)
1x new TrueNAS 13 U1 build on a Dell RX740XD with a Management NIC (bge0) and 2x iSCSI NICs going to iSCSI A (ix0) and iSCSI B. (ix1)

Block storage is setup like this.

vSphere I have added dynamic discovery to iSCSI HBA with 172.21.10.121:3260 - Rescanned storage, setup VMS6 Datastores for both the HDD extent and the SSD extent.

in static discovery, I can see the extent presented on both iSCSI A and B

in storage devices, I can see both my data stores.

I only have one path connected

 

[Murphy1138]

vSPhere continued - Only one path shown

All my other non TrueNAS SANs/NASs when configuered in the same way attach multi-path.

What trick am I missing here, any help would be greatly appriciated.

 

[sretalla]

The only thing jumping out at me is the use of the 27 bit subnet mask. I'm not sure if that will be handled properly from a routing perspective on all sides...

Maybe look into either testing with Class C or start the hunt in both directions for routing tables and possible clashes.

 

[HoneyBadger]

Weird. The /27 means you're not overlapping your subnets, so it's not the obvious one.

Questions:

Are you using port binding for the iSCSI vmks on the host?
Is the scenario the same on all three hosts (static discovery shows multiple paths, devices has a single path only)?
Can you vmkping from each of the A->A and B->B paths without issue?

 

[Murphy1138]

Weird. The /27 means you're not overlapping your subnets, so it's not the obvious one.

Questions:

Are you using port binding for the iSCSI vmks on the host?
Is the scenario the same on all three hosts (static discovery shows multiple paths, devices has a single path only)?
Can you vmkping from each of the A->A and B->B paths without issue?

Hi thank you for your reply and sorry for the late reply. Turns out it was the switch port the snd iSCSI B cable was going into to was wrong.

the vmkping showed down, letting a junior datacenter chap plumb in the cables thanks for the help!

 

[HoneyBadger]

Hi thank you for your reply and sorry for the late reply. Turns out it was the switch port the snd iSCSI B cable was going into to was wrong.

the vmkping showed down, letting a junior datacenter chap plumb in the cables thanks for the help!

Glad it was something minor. It's always worth working the OSI model from the bottom-up and asking "Is it plugged in?" before getting too deep anywhere else.

Assuming that's sorted - I assume you've done the necessary reading on how to stand TrueNAS up as a healthy and high-performance VM storage array? Specifically, there's a couple resources regarding sync write behavior (it needs to be forced to "always on" for the safety of your iSCSI data) and the preference for mirrors over RAIDZ

Resource - Sync writes, or: Why is my ESXi NFS so slow, and why is iSCSI faster?

This post is not specific to ESXi, however, ESXi users typically experience more trouble with this topic due to the way VMware works. When an application on a client machine wishes to write something to storage, it issues a write request. On a...

www.truenas.com

Some differences between RAIDZ and mirrors, and why we use mirrors for block storage

ZFS is a complicated, powerful system. Unfortunately, it isn't actually magic, and there's a lot of opportunity for disappointment if you don't understand what's going on. RAIDZ (including Z2, Z3) is good for storing large sequential files...

www.truenas.com

The R740XD is a fine starting platform certainly (any of the Dell XD's were, really) but there are a few caveats to watch out for, and if you're just getting started with this, it's best to nip them in the bud before loading a bunch of production data onto it.

 

[Murphy1138]

Glad it was something minor. It's always worth working the OSI model from the bottom-up and asking "Is it plugged in?" before getting too deep anywhere else.

Assuming that's sorted - I assume you've done the necessary reading on how to stand TrueNAS up as a healthy and high-performance VM storage array? Specifically, there's a couple resources regarding sync write behavior (it needs to be forced to "always on" for the safety of your iSCSI data) and the preference for mirrors over RAIDZ

Resource - Sync writes, or: Why is my ESXi NFS so slow, and why is iSCSI faster?

This post is not specific to ESXi, however, ESXi users typically experience more trouble with this topic due to the way VMware works. When an application on a client machine wishes to write something to storage, it issues a write request. On a...

www.truenas.com

Some differences between RAIDZ and mirrors, and why we use mirrors for block storage

ZFS is a complicated, powerful system. Unfortunately, it isn't actually magic, and there's a lot of opportunity for disappointment if you don't understand what's going on. RAIDZ (including Z2, Z3) is good for storing large sequential files...

www.truenas.com

The R740XD is a fine starting platform certainly (any of the Dell XD's were, really) but there are a few caveats to watch out for, and if you're just getting started with this, it's best to nip them in the bud before loading a bunch of production data onto it.

Thanks for the response and the links, that's my evening sorted out. Brilliant.

 

[HoneyBadger]

Thanks for the response and the links, that's my evening sorted out. Brilliant.

Apologies if I've given you a bunch of extra homework at the end of class.

Feel free to ask more questions, that's what we're here for.