PCI Passthrough and "still waiting for xpt_config"

[Elegant]

Hi guys, I recently installed FreeNAS on XenServer and am having issues getting FreeNAS to start when my LSI 9300-8i is passed through. I get the error found below:

Code:

run_interrupt_driven_hooks: still waiting after 60 seconds for xpt_config

This is preceeded by many of the following messages before finally kicking in:

Code:

mpr0: Warning: io_cmds_active is out of sync - resynching to 0

I can see that it finds and identifies all the drives attached to the card it just stays at the above message thereafter. If I disable the passthrough I can boot normally. Anyone have any ideas what can cause this? Am I missing a tunable? I've tried many versions of the firmware for my LSI card (P9-P12) and the error has appeared on all 4.

 

[jgreco]

We don't recommend virtualizing FreeNAS.

If you absolutely positively must virtualize FreeNAS, we suggest you use a hypervisor platform known to behave correctly. Currently that list includes ESXi 4.1, 5.5, and 6. It does not include Xen, since in the three years since I wrote that sticky, no one has demonstrated a stable and reliable Xen implementation that I'm aware of.

People periodically try other hypervisors such as KVM but invariably seem to end up with problems.

It is likely that there's still some issue with MSI/MSIX handling. I believe that's the usual cause of what you describe. Ditch the Xen and install ESXi.

 

[Elegant]

Ditch the Xen and install ESXi.

I would (and was) but I am looking to experiment with XenServer since it will actually let me passthrough my GPU (something ESXi would rather turn off natively while in a VM, no joke). Of course this GPU is not for the FreeNAS VM. In any case, I have found the following thread after much googling (didn't expect it to be here).

 

[jgreco]

Well, you may be in for a bumpy ride. Good luck.

 

[Elegant]

Thanks, I just resolved it, took a few tests but all is well. Your comment about MSI/MSIX handling was very useful!

 

[jgreco]

You would be well advised to test it for a minimum of several months and some severe lab testing before even considering trusting the thing with any real data.

https://forums.freenas.org/index.ph...-lsi-controllers-and-msi-x.42795/#post-281063

People have a tendency to assume that "yay it works" but "works" is something that's a long-term analysis. It took me at least a year or two to be satisfied that FreeNAS was stable on ESXi.