Expected data direction 2 for opcode ATA PASS-THROUGH(16) messages

[klosz007]

Hi,

I have been seeing these messages in console/dmesg output ever since I'm trying to use TNS as iSCSI target for my Win 10 PC:

Code:

[75619.447642] Failed CDB:
[75619.450660]  (h)___0__1__2__3__4__5__6__7__8__9__A__B__C__D__E__F
[75619.454198]    0: 85 08 0c 00 d0 00 01 00 01 00 4f 00 c2 00 b0 00   ..........O.....
[75619.459906] [9216]: scst: ***ERROR***: Expected data direction 2 for opcode ATA PASS-THROUGH(16) (handler vdisk_blockio, target iscsi) doesn't match decoded value 4

They appear periodically (think ~once a minute but nothing regular) when my Windows 10 iSCSI initiator connects to my TNS server and they immediately stop when it disconnects. I haven't seen these at all until I started running iSCSI target on TNS.

I couldn't google anything useful about these messages.

Hardware:
4-core i3-10300K + ASUS Z490G M/B
Intel X710-DA2 (in my PC and for TNS)
2x32GB+2x16GB DDR4 sticks

Initially I ran TNS on bare metal, currently (hardware consolidation) it is ESXi 7.0U3 VM with SAS disk controller redirected to VM.

Pool that iSCSI volumes reside on is RAIDZ1 on 4x Crucial MX500 2TB SSD's. They were initially connected to onboard Intel SATA controller, when I was running TNS on bare metal.
When I made it a VM, they are now connected to LSI 9300-8i (reflashed IBM M1215) which is passthorugh'ed to VM.

On TNS VM I use vmxnet3 + vSwitch. I tried to use SR-IOV too but then I had some kernel panics related to iSCSI under heavy iSCSI load so went back to using vmxnet3.

I have another RAIDZ2 pool on 8TB HDDs but this is for CIFS/SMB resources only.

TNS migration to VM and all these configuration changes mentioned above have not really changed anything in respect of these messages.
These messages were present before and after that and appear only when iSCSI client is connected to TNS target.

Can someone explain what they mean and what can be done to get rid of these please ?

Thanks,
Zbyszek

 

[klosz007]

No one replied but I eventually found the root cause. It was my Windows 10 PC's fault. I googled/read somewhere that disk health scanning software may cause such messages.

Initially I thought it was Samsung Magician (to monitor Samsung SSD's) - miss.
Then I suspected LSI Storage Authority Software (I used to have LSI RAID controller on my PC once but I did not uninstall LSA) - another miss.
Third shot was a hit - Corsair iCUE software. Luckily I did not have to uninstall it (I have a lot of Corsair accessories in my PC) - there's an option in iCUE to disable storage monitoring widget. Once I disabled it, all these console messages on TrueNAS were gone.

So iCUE was doing something to iSCSI-attached LUNs that TrueNAS did not like.

 

[Ericloewe]

Wait, was iCUE trying to issue ATA SMART commands via iSCSI? Presumably using SATA encapsulation? That's hilarious, and Windows passes that through over iSCSI?! Surely not, what am I missing here?

 

[klosz007]

Not sure what iCUE was exactly doing but once I disabled "Storage Drive Widget" in iCUE and restarted iCUE service, these messages on TrueNAS console immediately went away (otherwise they repeated every 10-15 seconds). Side effect is I can no longer add my SSD's to iCUE's dashboard.

 

[klosz007]

I believe this link:
https://sourceforge.net/p/scst/mailman/message/33082236/
made me search for 'something' that periodically issues SMART commands to every disk it can find on my PC. It was clearly iCUE. And it seems that Windows iSCSI initiator passes these SMART commands down to TrueNAS.