Unable to create pool in Truenas Scale Proxmox VM

[epicurean]

Not sure if this is a proxmox question or a truenas one.
I passed through the entire HBA in IT mode to the Truenas VM in Proxmox 7.
However, I cannot create a new pool , or add a vdev in Truenas. I was able to do so just a week ago..until I updated proxmox recently, and also truenas scale a few days ago.

Consistently getting this error

Error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/middlewared/job.py", line 426, in run await self.future File "/usr/lib/python3/dist-packages/middlewared/job.py", line 461, in __run_body rv = await self.method(*([self] + args)) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1186, in nf res = await f(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1318, in nf return await func(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 757, in do_create verrors.check() File "/usr/lib/python3/dist-packages/middlewared/service_exception.py", line 70, in check raise self middlewared.service_exception.ValidationErrors: [EINVAL] pool_create.topology: Disks have duplicate serial numbers: '' (vda, vdb).

I googled this and found that most people have similar errors either only passthrough their drives, and not the entire HB or their drives were USB attached drives. Which is not true for my case

Any help appreciate. Much thanks!

 

[HoneyBadger]

Disks have duplicate serial numbers: '' (vda, vdb)

The vd* naming convention seems to imply that the disks are being handled through virtio and not through a regular SAS HBA driver. What is the model of the HBA you're passing through, and can you check your ProxMox VM .conf file to ensure it hasn't somehow lost the passthrough configuration?

 

[epicurean]

Thank you for your reply.
The HBAs are 2 x 9300-8i. This is the the conf file:

agent: 1
balloon: 0
bios: ovmf
boot: order=scsi0
cores: 2
hostpci0: 0000:03:00,pcie=1
hostpci1: 0000:04:00,pcie=1
hotplug: disk,usb
machine: q35
memory: 49152
meta: creation-qemu=7.1.0,ctime=1677309047
name: Truenas-x99
net0: virtio=D2:9E:99:FB:7B:A5,bridge=vmbr0,firewall=1
numa: 0
onboot: 1
ostype: l26
scsi0: Storage:222/vm-222-disk-0.qcow2,discard=on,iothread=1,size=40G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=eb4cf1bb-6434-4bc0-bcca-0f01c84ab565
sockets: 2
startup: order=1
virtio2: /dev/disk/by-id/wwn-0x5002538e00c3520a,backup=0,size=1875374424K
virtio3: /dev/disk/by-id/wwn-0x5002538e0002a00e,backup=0,size=1875374424K
vmgenid: 72da7c71-07f0-499f-a72c-e3e64fdda475

 

[HoneyBadger]

hostpci0: 0000:03:00,pcie=1
hostpci1: 0000:04:00,pcie=1
virtio2: /dev/disk/by-id/wwn-0x5002538e00c3520a,backup=0,size=1875374424K
virtio3: /dev/disk/by-id/wwn-0x5002538e0002a00e,backup=0,size=1875374424K

It looks like you've got both the correct PCIe stubbing (the hostpciX lines) as well as a pair of virtio disks there. It's the latter that's tripping you up, as the SCALE middleware will stop if it detects any disks with duplicate or absent serial numbers.

You'll need to append a serial number at the end of the line in the format of ,serial=XXXX-YYYY-YOURSERIAL on the end of each virtio disk line, using your own string for the serial number (perhaps the disk WWN?) such as:

Code:

virtio2: /dev/disk/by-id/wwn-0x5002538e00c3520a,backup=0,size=1875374424K,serial=5002538e00c3520a
virtio3: /dev/disk/by-id/wwn-0x5002538e0002a00e,backup=0,size=1875374424K,serial=5002538e0002a00e

You'll need to power off the VM entirely to get ProxMox to re-read the .conf file.

What are the virtio backed disks being used for?

 

[epicurean]

Thank you Sir!
Adding the serial numbers did the trick.
The virtio backed disks are just for another mirror pool for data - attached to the motherboard SATA ports.There was not enough on the HBA.

 

[HoneyBadger]

Glad it worked out!

The virtio backed disks are just for another mirror pool for data - attached to the motherboard SATA ports.There was not enough on the HBA.

Bear in mind that using virtual disks isn't recommended for production data, even in this "local passthrough" mode - the virtio driver and interaction with the host OS could still leave opportunities for a disagreement between ZFS/ProxMox/disk hardware as to what has or hasn't been successfully flushed.

 

[epicurean]

Glad it worked out!



Bear in mind that using virtual disks isn't recommended for production data, even in this "local passthrough" mode - the virtio driver and interaction with the host OS could still leave opportunities for a disagreement between ZFS/ProxMox/disk hardware as to what has or hasn't been successfully flushed.

Thank you! I will definately keep that in mind