Unable to pass PCIe SATA controller to VM <solved>

[graylion]

Hi, I am trying to access a group of disks from a former (dead) server in a VM. To this end I have procured a SATA controller and attached the disks to it. I have added the controller to the VM as PCI passthrough. when I try to boot the VM, I get

"middlewared.service_exception.CallError: [EFAULT] internal error: qemu unexpectedly closed the monitor: 2023-07-27T23:59:35.560753Z qemu-system-x86_64: -device vfio-pci,host=0000:04:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio 0000:04:00.0: group 8 is not viable Please ensure all devices within the iommu_group are bound to their vfio bus driver."

IOMMU group 8 is

Group 8: [1022:43d5] [R] 01:00.0 USB controller 400 Series Chipset USB 3.1 XHCI Controller
[1022:43c8] 01:00.1 SATA controller 400 Series Chipset SATA Controller
[1022:43c6] 01:00.2 PCI bridge 400 Series Chipset PCIe Bridge
[1022:43c7] [R] 02:00.0 PCI bridge 400 Series Chipset PCIe Port
[1022:43c7] 02:04.0 PCI bridge 400 Series Chipset PCIe Port
[1022:43c7] 02:05.0 PCI bridge 400 Series Chipset PCIe Port
[1022:43c7] 02:06.0 PCI bridge 400 Series Chipset PCIe Port
[1022:43c7] 02:07.0 PCI bridge 400 Series Chipset PCIe Port
[1b21:1064] [R] 04:00.0 SATA controller Device 1064
[10ec:8168] [R] 07:00.0 Ethernet controller RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller

lspci -v gives

04:00.0 SATA controller: ASMedia Technology Inc. Device 1064 (rev 02) (prog-if 01 [AHCI 1.0])
Subsystem: ZyDAS Technology Corp. Device 2116
Flags: fast devsel, IRQ 31, IOMMU group 8
Memory at fcd82000 (32-bit, non-prefetchable) [size=8K]
Memory at fcd80000 (32-bit, non-prefetchable) [size=8K]
Expansion ROM at fcd00000 [disabled] [size=512K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [80] Express Endpoint, MSI 00
Capabilities: [100] Advanced Error Reporting
Capabilities: [130] Secondary PCI Express
Kernel driver in use: vfio-pci
Kernel modules: ahci


Version:
TrueNAS-SCALE-22.12.3.3

help?

edit - is there a method that does not require the ACS patch which seems not to be in Scale?

Many thanks in advance

 

[graylion]

nevermind. I mounted the VG under the host OS and copied the data from the commandline.

 

[NickF]

What are your system specs? It's all guesswork unless you give us some more information about what you are working with.

Also, as a general rule we try to shy folks away from cheap SATA cards. In your example, I would hold still to that wisdom even if you are passing it to another OS. The same goes for Realtek NIC cards, while they generally work, they can be inconsistent in their performance and used to suffer from horrible drivers. I don't think that applies here in Linux-land, at least not as much as in FreeBSD, but I am also not sure. I've rigged up some test boxes for fun with Aquantia and Realtek NICs for cluster on SCALE but I'm not sure I would trust them for a real workload.

 

[graylion]

AMD Ryzen 5 5600G
128 GiB of RAM
mATX board

 

[NickF]

What motherboard?

 

[graylion]

ASUS PRIME B450M-A II

 

[ChrisRJ]

As @NickF already mentioned there is a ton of details missing. If the problem was already solved, that is great. In this case, marking the thread as solved would be great.

In particular I did not see the virtualization software mentioned. This is absolutely vital information.

 

[graylion]

Truenas Scale, so it is KVM. Client OS is Linux Mint.