samurai336
Cadet
- Joined
- Mar 23, 2022
- Messages
- 2
Good afternoon the tl;dr is that I cannot figure out why the PCI pass through in my TrueNAS scale install has stopped working. I've tried a lot of different things including reinstalling TrueNAS scale, moving the card to a different PCI slot and about a dozen other things which I have described in detail below.
I have also described all the different things I did that led up to this issue and all the solutions I've tried so far to diagnose and remedy. At this point though I have no idea why its now working.
--Back story--
In the beginning I had an old daily driving PC I replaced with some fairly good specs. It has an intel i7-4770 CPU and 32GB of ram. I built it as a gaming PC back in 2013 that I was using up until a few months ago when I replaced it after it had some issues with handling a newer graphics card.
Not wanting to throw it out or have it become a dust collector, I put linux mint on there bought a TV tuner card (Hauppauge 1196 WinTV HVR-1265 PCI Express), attached an antenna to it and hooked it up to my living rooms TV, with Plex and nextPVR. This worked great as a little media server for several months in my living room with no problems.
However, inspired to degooglify my life I decided it might be better to turn that machine into something that could handle such things better and I had read and heard about TrueNAS scale. In my research I saw that while it was not quite there to pass hardware through to a docker image, a Virtual Machine should be no problem. So I decided to get some extra HDDs and install TrueNAS scale.
--Discovering the Problem--
I got TrueNAS scale up and running on this machine very quickly and set up my first pool. After tooling around with the provided Plex app I decided a VM is what I really needed. So I set up an ubuntu server vm in TrueNAS. I tried to add hardware but the list was empty. After some research(https://www.truenas.com/community/t...ie-device-such-as-a-network-card-to-vm.95635/) I discovered there may be some settings not enabled on my motherboard. So into the bios I went, virtualization was enabled but then I dug around and found that vt-d was not enabled so I turned that on and booted back into TrueNAS. I was then able to see all the PCI devices and could add them to the VM.
Using `lspci` I found the TV tuner card and made sure to add that as a PCI passthrough device in the VM settings then booted up the VM, installed nextPVR and did a channel scan and sure enough I had broadcast TV coming through the VM to my web browser on my other desktop.
From here I went to set up the NFS mounting for this media server VM, but to do that I had to set up a network bridge. As an aside I must say that following these directions from TrueNAS was a complete PITA and required several reboots to get changes to stick (https://www.truenas.com/docs/scale/virtualization/accessingnasfromvm/). Finally the VM could ping the internet and the host machine.
At this point I wanted to add another pair of HDDs to the machine so I shut the machine down and plugged an additional pair of HDDs. Turned it back on and created a second Pool with them.
Happy with my creation, I decided to double check the media servers nextPVR instance to see if the channel guide had updated and when I clicked to watch a channel at the bottom I saw `Streaming Failed (transcoder exited)`. I slowly went through rebooting everything; the VM, the machine TrueNAS was on but kept running into the same issue.
--Diagnosis–
It was previously working. I had seen it with my own eyes, so I wasn’t sure what had changed. I started with the last non hardware thing I did which was the network bridge. So I deleted that and put it back to how it was and that did not solve the problem.
I then suspected it was a power draw problem so I opened the machine back up and unplugged the power from the two additional HDDs I added and the same problem when I booted back up.
Next to make sure it was not an actual hardware problem, I took a USB of Linux Lite and I booted that and installed nextPVR and told it to channel scan and it worked and I was once again able to stream live TV to my web browser. So the hardware was fine.
Next I moved the TV tuner PCI card to a different PCI slot and changed the VM to point to the new PCI address. Same problem.
Next I spun up a new VM and added the TV cards PCI address as a passthrough to that new VM and installed nextPVR there and still the same problem.
Next I started looking into dmesg to see if there were any problems. No errors reported.
: 0xf0800000
No errors reported.
The latest thing I’ve done is replace the battery on my machine's motherboard and after it reset the CMOS, turned virtualization and vt-d back on, and checked it turned on correctly which it looks like it did.
And still nothing. Finally I thought I’d pull out a big hammer and reinstall the TrueNAS scale. Which I did but I didn’t nuke my old settings (it did not seem to give me the option) this did not work but I do wonder if I should escalate to that.
--conclusion–
So I am stumped both on what could be wrong and how to diagnose this.
Please let me know your thoughts or any additional information I might be able to provide
Thanks
I have also described all the different things I did that led up to this issue and all the solutions I've tried so far to diagnose and remedy. At this point though I have no idea why its now working.
--Back story--
In the beginning I had an old daily driving PC I replaced with some fairly good specs. It has an intel i7-4770 CPU and 32GB of ram. I built it as a gaming PC back in 2013 that I was using up until a few months ago when I replaced it after it had some issues with handling a newer graphics card.
Not wanting to throw it out or have it become a dust collector, I put linux mint on there bought a TV tuner card (Hauppauge 1196 WinTV HVR-1265 PCI Express), attached an antenna to it and hooked it up to my living rooms TV, with Plex and nextPVR. This worked great as a little media server for several months in my living room with no problems.
However, inspired to degooglify my life I decided it might be better to turn that machine into something that could handle such things better and I had read and heard about TrueNAS scale. In my research I saw that while it was not quite there to pass hardware through to a docker image, a Virtual Machine should be no problem. So I decided to get some extra HDDs and install TrueNAS scale.
--Discovering the Problem--
I got TrueNAS scale up and running on this machine very quickly and set up my first pool. After tooling around with the provided Plex app I decided a VM is what I really needed. So I set up an ubuntu server vm in TrueNAS. I tried to add hardware but the list was empty. After some research(https://www.truenas.com/community/t...ie-device-such-as-a-network-card-to-vm.95635/) I discovered there may be some settings not enabled on my motherboard. So into the bios I went, virtualization was enabled but then I dug around and found that vt-d was not enabled so I turned that on and booted back into TrueNAS. I was then able to see all the PCI devices and could add them to the VM.
Using `lspci` I found the TV tuner card and made sure to add that as a PCI passthrough device in the VM settings then booted up the VM, installed nextPVR and did a channel scan and sure enough I had broadcast TV coming through the VM to my web browser on my other desktop.
From here I went to set up the NFS mounting for this media server VM, but to do that I had to set up a network bridge. As an aside I must say that following these directions from TrueNAS was a complete PITA and required several reboots to get changes to stick (https://www.truenas.com/docs/scale/virtualization/accessingnasfromvm/). Finally the VM could ping the internet and the host machine.
At this point I wanted to add another pair of HDDs to the machine so I shut the machine down and plugged an additional pair of HDDs. Turned it back on and created a second Pool with them.
Happy with my creation, I decided to double check the media servers nextPVR instance to see if the channel guide had updated and when I clicked to watch a channel at the bottom I saw `Streaming Failed (transcoder exited)`. I slowly went through rebooting everything; the VM, the machine TrueNAS was on but kept running into the same issue.
--Diagnosis–
It was previously working. I had seen it with my own eyes, so I wasn’t sure what had changed. I started with the last non hardware thing I did which was the network bridge. So I deleted that and put it back to how it was and that did not solve the problem.
I then suspected it was a power draw problem so I opened the machine back up and unplugged the power from the two additional HDDs I added and the same problem when I booted back up.
Next to make sure it was not an actual hardware problem, I took a USB of Linux Lite and I booted that and installed nextPVR and told it to channel scan and it worked and I was once again able to stream live TV to my web browser. So the hardware was fine.
Next I moved the TV tuner PCI card to a different PCI slot and changed the VM to point to the new PCI address. Same problem.
Next I spun up a new VM and added the TV cards PCI address as a passthrough to that new VM and installed nextPVR there and still the same problem.
Next I started looking into dmesg to see if there were any problems. No errors reported.
Code:
[ 5.967675] tveeprom: Hauppauge model 161111, rev A1I6, serial# 4036167623 [ 5.967676] tveeprom: MAC address is 00:0d:fe:93:07:c7 [ 5.967677] tveeprom: tuner model is SiLabs Si2157 (idx 186, type 4) [ 5.967677] tveeprom: TV standards NTSC(M) ATSC/DVB Digital (eeprom 0x88) [ 5.967678] tveeprom: audio processor is CX23888 (idx 40) [ 5.967678] tveeprom: decoder processor is CX23888 (idx 34) [ 5.967679] tveeprom: has no radio, has IR receiver, has no IR transmitter [ 5.967679] cx23885: cx23885[0]: hauppauge eeprom: model=161111 [ 5.986754] Console: switching to colour frame buffer device 240x67 [ 6.011266] amdgpu 0000:01:00.0: [drm] fb0: amdgpudrmfb frame buffer device [ 6.036847] cx25840 8-0044: cx23888 A/V decoder found @ 0x88 (cx23885[0]) [ 6.041001] b43-phy0: Broadcom 4352 WLAN found (core revision 42) [ 6.041634] b43-phy0 ERROR: FOUND UNSUPPORTED PHY (Analog 12, Type 11 (AC), Revision 1) [ 6.041987] b43: probe of bcma0:1 failed with error -95 [ 6.042221] Broadcom 43xx driver loaded [ Features: PNLS ] [ 6.049063] [drm] Initialized amdgpu 3.40.0 20150101 for 0000:01:00.0 on minor 0 [ 6.668160] cx25840 8-0044: loaded v4l-cx23885-avcore-01.fw firmware (16382 bytes) [ 6.697143] intel_rapl_common: Found RAPL domain package [ 6.697376] intel_rapl_common: Found RAPL domain core [ 6.697590] intel_rapl_common: Found RAPL domain dram [ 6.748200] cx23885: cx23885[0]: registered device video0 [v4l2] [ 6.748484] cx23885: cx23885[0]: registered device vbi0 [ 6.748807] cx23885: cx23885[0]: alsa: registered ALSA audio device [ 6.748809] cx23885: cx23885_dvb_register() allocating 1 frontend(s) [ 6.749080] cx23885: cx23885[0]: cx23885 based dvb card [ 6.761028] si2157 7-0060: Silicon Labs Si2147/2148/2157/2158 successfully attached [ 6.761353] dvbdev: DVB: registering new adapter (cx23885[0]) [ 6.761595] cx23885 0000:06:00.0: DVB: registering adapter 0 frontend 0 (LG Electronics LGDT3306A VSB/QAM Frontend)... [ 6.762261] cx23885: cx23885_dev_checkrevision() Hardware revision = 0xd0 [ 6.762559] cx23885: cx23885[0]/0: found at 0000:06:00.0, rev: 4, irq: 16, latency: 0, mmio
: 0xf0800000
No errors reported.
The latest thing I’ve done is replace the battery on my machine's motherboard and after it reset the CMOS, turned virtualization and vt-d back on, and checked it turned on correctly which it looks like it did.
Code:
dmesg | grep -i -e DMAR -e IOMMU [ 0.000000] Command line: BOOT_IMAGE=/ROOT/22.02.RELEASE@/boot/vmlinuz-5.10.93+truenas root=ZFS=boot-pool/ROOT/22.02.RELEASE ro console=ttyS0,9600 console=tty1 libata.allow_tpm=1 systemd.unified_cgroup_hierarchy=0 amd_iommu=on iommu=pt kvm_amd.npt=1 kvm_amd.avic=1 intel_iommu=on zfsforce=1 [ 0.007451] ACPI: DMAR 0x00000000DD955738 000080 (v01 INTEL HSW 00000001 INTL 00000001) [ 0.007467] ACPI: Reserving DMAR table memory at [mem 0xdd955738-0xdd9557b7] [ 0.018013] Kernel command line: BOOT_IMAGE=/ROOT/22.02.RELEASE@/boot/vmlinuz-5.10.93+truenas root=ZFS=boot-pool/ROOT/22.02.RELEASE ro console=ttyS0,9600 console=tty1 libata.allow_tpm=1 systemd.unified_cgroup_hierarchy=0 amd_iommu=on iommu=pt kvm_amd.npt=1 kvm_amd.avic=1 intel_iommu=on zfsforce=1 [ 0.018097] DMAR: IOMMU enabled [ 0.091269] DMAR: Host address width 39 [ 0.091410] DMAR: DRHD base: 0x000000fed90000 flags: 0x1 [ 0.091604] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap d2008c20660462 ecap f010da [ 0.091888] DMAR: RMRR base: 0x000000dd8c8000 end: 0x000000dd8d4fff [ 0.092114] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed90000 IOMMU 0 [ 0.092343] DMAR-IR: HPET id 0 under DRHD base 0xfed90000 [ 0.092539] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping. [ 0.093041] DMAR-IR: Enabled IRQ remapping in x2apic mode [ 0.423257] iommu: Default domain type: Passthrough (set via kernel command line) [ 1.176592] DMAR: No ATSR found [ 1.176749] DMAR: dmar0: Using Queued invalidation [ 1.176967] pci 0000:00:00.0: Adding to iommu group 0 [ 1.177164] pci 0000:00:01.0: Adding to iommu group 1 [ 1.177357] pci 0000:00:14.0: Adding to iommu group 2 [ 1.177552] pci 0000:00:16.0: Adding to iommu group 3 [ 1.177744] pci 0000:00:19.0: Adding to iommu group 4 [ 1.177937] pci 0000:00:1a.0: Adding to iommu group 5 [ 1.178129] pci 0000:00:1b.0: Adding to iommu group 6 [ 1.178321] pci 0000:00:1c.0: Adding to iommu group 7 [ 1.178515] pci 0000:00:1c.1: Adding to iommu group 8 [ 1.178707] pci 0000:00:1c.3: Adding to iommu group 9 [ 1.178900] pci 0000:00:1d.0: Adding to iommu group 10 [ 1.179104] pci 0000:00:1f.0: Adding to iommu group 11 [ 1.179299] pci 0000:00:1f.2: Adding to iommu group 11 [ 1.179538] pci 0000:00:1f.3: Adding to iommu group 11 [ 1.179730] pci 0000:01:00.0: Adding to iommu group 1 [ 1.179917] pci 0000:01:00.1: Adding to iommu group 1 [ 1.180110] pci 0000:03:00.0: Adding to iommu group 12 [ 1.180305] pci 0000:04:00.0: Adding to iommu group 13 [ 1.180500] pci 0000:05:01.0: Adding to iommu group 14 [ 1.180695] pci 0000:05:04.0: Adding to iommu group 15 [ 1.180894] pci 0000:05:05.0: Adding to iommu group 16 [ 1.181089] pci 0000:05:06.0: Adding to iommu group 17 [ 1.181284] pci 0000:05:07.0: Adding to iommu group 18 [ 1.181479] pci 0000:05:08.0: Adding to iommu group 19 [ 1.181674] pci 0000:05:09.0: Adding to iommu group 20 [ 1.181872] pci 0000:06:00.0: Adding to iommu group 21 [ 1.182069] pci 0000:08:00.0: Adding to iommu group 22 [ 1.182266] pci 0000:0a:00.0: Adding to iommu group 23 [ 1.182463] pci 0000:0c:00.0: Adding to iommu group 24 [ 1.182701] DMAR: Intel(R) Virtualization Technology for Directed I/O [ 1.197929] AMD-Vi: AMD IOMMUv2 functionality not available on this system - This is not a bug. [ 1.340976] intel_iommu=on
And still nothing. Finally I thought I’d pull out a big hammer and reinstall the TrueNAS scale. Which I did but I didn’t nuke my old settings (it did not seem to give me the option) this did not work but I do wonder if I should escalate to that.
--conclusion–
So I am stumped both on what could be wrong and how to diagnose this.
Please let me know your thoughts or any additional information I might be able to provide
Thanks