Can't configure VMFS6 over iSCSI

[skyforest]

Hi all!

I have a 2x TrueNAS scale server implementation with iSCSI VMFS6 datastores configured for ESXi. Both servers are running the latest Scale release (22.02 RC2). Due to something completely unrelated to VMWare, I wound up rebuilding one of the servers and re-configured everything from scratch. I configured the iSCSI share identically to the way they were before, and identically to how they are configured on the second server. The ESXi hosts can connect to and read the volumes just fine, but when I try to partition and format with VMFS6, it fails (both through the vSphere GUI and the CLI). Oddly, I can format the partition with VMFS5.

For reference, here is the command I'm trying to execute on one of the hosts:

vmkfstools -C vmfs6 -S Sequoia-1 /vmfs/devices/disks/naa.6589cfc000000848faaf347d35569687:1
create fs deviceName:'/vmfs/devices/disks/naa.6589cfc000000848faaf347d35569687:1', fsShortName:'vmfs6', fsName:'Sequoia-1
deviceFullPath:/dev/disks/naa.6589cfc000000848faaf347d35569687:1 deviceFile:naa.6589cfc000000848faaf347d35569687:1
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Scanning for VMFS-3/VMFS-5 host activity (512 bytes/HB, 2048 HBs).
Creating vmfs6 file system on "naa.6589cfc000000848faaf347d35569687:1" with blockSize 1048576, unmapGranularity 1048576, unmapPriority default and volume label "Sequoia-1".
Failed to create VMFS on device naa.6589cfc000000848faaf347d35569687:1
Usage: vmkfstools -C [vmfs5|vmfs6|vfat] /vmfs/devices/disks/vml... or,
vmkfstools -C [vmfs5|vmfs6|vfat] /vmfs/devices/disks/naa... or,
vmkfstools -C [vmfs5|vmfs6|vfat] /vmfs/devices/disks/mpx.vmhbaA:T:L:P
Error: Invalid argument

I've come across some similar issues on the VMWare forums, check out this thread for reference: https://communities.vmware.com/t5/E...see-VMkernel-log-for-more/m-p/2885426#M279552

It looks like most of those issues were related to incorrectly configured iSCSI shares using file-based extents rather than zVol's - which isn't the issue here. As mentioned above, the configuration is identical to the other server that's working just fine. Could this be something specific to the latest TrueNAS release?

Please let me know if there's anything else I can provide. Thank you in advance for any assistance!

 

[morganL]

TrueNAS Enterprise (and by implication CORE) is VMware certified and robust.
TrueNAS SCALE has to use a different iSCSI stack due to Linux. This has not yet gone through a VMware certification.
So, users with VMware needs should probably stick with CORE unless there is a desire to be part of the development and test team.
We'll be doing more extensive iSCSI testing in january and February.

Feel free to document and report any bugs on a single SCALE system. There should be nothing special about having two systems and no support for clustered ISCSI in the first Angelfish release.

 

[skyforest]

Hi Morgan

Thanks for the quick response, all makes sense. For what it’s worth, the iscsi implementation that has been working to date on Scale was an imported volume from a previous Core system. The one I’m having issues with now is a new volume created in Scale.

Happy to help with any testing and bug reporting - is this the best place to report issues?

Kind regards

Graham

 

[Murphy1138]

Hi, I had this same issue. It was the block size on the extent. It was set to 128k and had to reduce to 64k then it worked.

 

[ericsmith881]

Hi, I had this same issue. It was the block size on the extent. It was set to 128k and had to reduce to 64k then it worked.

I've tried this with block size 64k and VMware still refuses to format it as a datastore. I've also tried the old trick from CORE by checking "disable physical block size reporting." That didn't work either.

I'm giving up for now. It looks like SCALE isn't ready for use with VMware.

 

[morganL]

I've tried this with block size 64k and VMware still refuses to format it as a datastore. I've also tried the old trick from CORE by checking "disable physical block size reporting." That didn't work either.

I'm giving up for now. It looks like SCALE isn't ready for use with VMware.

Can you be a little more clear.

What was iSCSI block size?

What was zvol block size?

There might be a related issue reported here. https://www.truenas.com/community/threads/iscsi-block-size-reporting-issues-on-22-12.107717/