SOLVED Issues with 9211-8i PCI passthrough on Xen 7

[Nick R]

I have a host running XenServer 7 on a SuperMicro X10SDV-TLN4F (rev 2.0 Xeon D-1941). I am able to passthrough the Intel Lynx-Point SATA controller and both FreeNAS 9.10 and 10-ALPHA2 see it fine and is usable. I also have a 9211-8i in IT mode running 20.00.07.00 (reflash from a IBM M1015) but when I passthrough the 9211-8i adapter in the FreeNAS vm, I am getting numerous mps0: Warning: io_cmds_active is out of sync - resynching to 0 then eventually it sees the 8 SATA disks attached then finally it just hangs.

Here is the kicker, it works fine in ESXi 6.0U2 and FreeNAS sees everything fine. Only quirk is for the onboard SATA interface, you need to add the following in /etc/vmware/passthrough.map:
# INTEL Lynx Point AHCI
8086 8c02 d3d0 false

I have tried to add mpt2sas.msix_disable=1 or mpt3sas.msix_disable=1 to Xen's grub.cfg but it changes nothing. Not sure what it could be at this point, but I may have to switch to ESXi although would prefer to stay with Xen.

 

[Nick R]

Well... figured it out myself. Had to add
set.kFreeBSD.hw.mps.disable_msix=1 in the VM's boot commands in grub (to install properly) then add hw.mps.disable_msix=1 in /boot/loader.conf

Now it works (at least right now, will report back if long-term issues show up).

 

[Dice]

What's the follow up?