Cobia not allowing FFmpeg and Node.js to be installed directly.

[littleNewton]

Install a VM and use that as you see fit.

My TrueNAS SCALE is a VM that runs on ESXi. A lot of PCIe devices were passthrough to this VM.

I have considered using jail or container which can provide the software I needed. However, the memory management of TrueNAS SCALE, or OpenZFS on Linux, is not very robust. Linux VFS subsystem and ZFS arc have many similar functionalities. Without adjustment of zfs_max_arc, ARC can only utilize half of my 512GB memory which is unacceptable. Under this situation, I don't want to run a container with --daemon on my back machine.

 

[Etorix]

My TrueNAS SCALE is a VM that runs on ESXi.

You really, really should have stated it in the first post.
Then, if you understand the pitfalls of virtualising TrueNAS, and are aware of the issue with ARC on SCALE, use TrueNAS CORE instead for ZFS storage and do your transcoding in another ESXi VM.
If you need containers, set up a Linux VM in ESXi and install docker.

 

[littleNewton]

You really, really should have stated it in the first post.
Then, if you understand the pitfalls of virtualising TrueNAS, and are aware of the issue with ARC on SCALE, use TrueNAS CORE instead for ZFS storage and do your transcoding in another ESXi VM.
If you need containers, set up a Linux VM in ESXi and install docker.

Yes, My ESXi hypervisor have 3 VMs:

• Windows Server for downloading
• TrueNAS SCALE for storage
• Debian for general purpose

Those 3 VMs are fabricated with vSwitch of ESXi. Each of the VMs are assigned with a Mellanox CX4 Lx SR-IOV VF.

The bandwidth of vSwtich is 24Gbps.

Running those applications on Debian is considerable. However, the vast traffic on vSwitch will slow down my whole system.

 

[Davvo]

You could merge the TN and the WS istances with CORE + JAILS, easing the burden on the vSwitch using mounting points.

 

[Whattteva]

My TrueNAS SCALE is a VM that runs on ESXi. A lot of PCIe devices were passthrough to this VM.

I have considered using jail or container which can provide the software I needed. However, the memory management of TrueNAS SCALE, or OpenZFS on Linux, is not very robust. Linux VFS subsystem and ZFS arc have many similar functionalities. Without adjustment of zfs_max_arc, ARC can only utilize half of my 512GB memory which is unacceptable. Under this situation, I don't want to run a container with --daemon on my back machine.

Since you're running as a VM, why not just run CORE? Most people run SCALE for the Apps, but since you're running this as a VM on a hypervisor, you kinda' have no need for it since you can just spin another VM that would be better-suited to handle such things than SCALE. CORE has no such limitation on ZFS ARC and can and will use up to <1 GB than your allocated RAM.

In fact, that's exactly what I did with my setup, except that I'm using Proxmox as the hypervisor instead of ESXi. I have a vanilla FreeBSD VM + jails where I install all my services.

 

[littleNewton]

Since you're running as a VM, why not just run CORE? Most people run SCALE for the Apps, but since you're running this as a VM on a hypervisor, you kinda' have no need for it since you can just spin another VM that would be better-suited to handle such things than SCALE. CORE has no such limitation on ZFS ARC and can and will use up to <1 GB than your allocated RAM.

In fact, that's exactly what I did with my setup, except that I'm using Proxmox as the hypervisor instead of ESXi. I have a vanilla FreeBSD VM + jails where I install all my services.

Yeah. You are definitely right.

Firstly, I tried to use CORE as my storage server. I passed through an HBA (SAS3008) card to it. When I boot the VM, I always stuck on a stage that is related to the HBA. I didn't know why.

After trying many times, I gave up CORE.

PS. The WebUI of CORE is a little bit uglier than SCALE.

 

[Whattteva]

Yeah. You are definitely right.

Firstly, I tried to use CORE as my storage server. I passed through an HBA (SAS3008) card to it. When I boot the VM, I always stuck on a stage that is related to the HBA. I didn't know why.

Ah, I ran into this issue also. You probably need to turn off the OPROM so it can get past the boot process. That OPROM really is only needed if you need to boot from the HBA, which I'm guessing you don't need to since you're probably using a virtual disk as your boot device. Also, make sure you're running the latest version of the firmware. Not sure for the SAS3008, but for my SAS2308, it's 20.00.07.00.

PS. The WebUI of CORE is a little bit uglier than SCALE.

I guess I can agree with the aesthetics, but in my opinion, CORE UI is more usable since some menus just stay open and not force me to keep opening it when I hop between different screens like SCALE forces me to do.

 

[littleNewton]

I'm guessing you don't need to

Yes, I do really have a virtual boot disk.

I am trying to install a CORE now!!!

 

[littleNewton]

turn off the OPROM so it can get past the boot process

Would you mind telling me how to disable the OPROM?

 

[littleNewton]

You probably need to turn off the OPROM so it can get past the boot process.

OK, I am trying to disable it.

 

[morganL]

Hi danb35,

I want to use ffmpeg and node.js. These two are common software.

Given that they can work properly on the elder version of TrueNAS SCALE, it is reasonable that they still work on the newer release.

This is not a feature request but a bug report, especially the ffmpeg one. As we all know, Cobia is still an RC version, not a release one.

Best,
Peng

It may or may not be a bug..
With new releases, there is a continual goal to increase reliability, quality and security..... sometimes that means the software may become more restricted. In general, we discourage adding software directly since it is untested and may have a conflict with other software.

The key challenge is getting agreement on a use-case...and what is the best way to deploy. Should ffmpeg be in the base software or should it be in an App?

So, I'd suggest leading with a thread in the Cobia subforum... describing what you are trying to achieve and what you did in Bluefin. There can then be a discussion on how best to meet your requirements.

I'm going to edit the tread title to be more specific to the problem..... its not a general RC.1 issue.

 

[Whattteva]

Would you mind telling me how to disable the OPROM?

It should be a setting in your BIOS.

View attachment 70571

OK, I am trying to disable it.

Ah yeah, that should do it, hopefully.

 

[littleNewton]

It should be a setting in your BIOS.

Ah yeah, that should do it, hopefully.

I have turned it off. But it would still stuck in a state which is related to MPR0 device.

 

[Whattteva]

I have turned it off. But it would still stuck in a state which is related to MPR0 device.

I'm sorry, I'm afraid I may not be able to help you further. Seems like our devices differ. Mine is mps0 instead of mpr0 and seems to boot just fine. Could you perhaps post a screenshot of where it gets stuck?

 

[littleNewton]

I'm sorry, I'm afraid I may not be able to help you further. Seems like our devices differ. Mine is mps0 instead of mpr0 and seems to boot just fine. Could you perhaps post a screenshot of where it gets stuck?

Boot Stuck mps0: Reinitializing controller.

But it would still stuck in a state which is related to MPR0 device.

It was not MPR0. I am sorry for providing the wrong message.

 

[Whattteva]

Boot Stuck mps0: Reinitializing controller.


It was not MPR0. I am sorry for providing the wrong message.

Just want to drop this here for your reference. It seems to be related particularly with ESXi passthrough and FreeBSD 13. Seems like others have had success either dropping back to 12.2 or swapping to SAS2308. They filed bug tickets with both FreeBSD and ixSystems, so hopefully this will be resolved in the future. Sorry, not quite the solution you probably were hoping for.

Boot Stuck "mps0: Reinitializing controller" after 13.0-U3 upgrade

System & History: Freenas/TrueNAS virtualized on ESXi Dell H310 (sas2008) Flashed to ITmode FW ver 20.00.07.00 passed through to freenas VM System has been running stable for 4 years across the following versions: 11.1u4 -> 11.1u6 -> 11.1u7 - >11.3u5 -> 12.2u8.1 Today I went to upgrade to...

www.truenas.com

 

[katbyte]

as of TrueNAS-SCALE-24.04-RC.1 ffmpeg still doesn't work when installed via apt:

ffprobe: error while loading shared libraries: libGL.so.1: cannot open shared object file: No such file or directory

I would also consider this a regression as it used to work and now prevents people from installing ffmpeg

yes yes "appliance" and all that but its also just linux and imho it should be expected people want or need to install things as the base system is missing a number of important/useful binaries imho or they are out of date like storcli (currently doesn't support /cx show temperature)

as for why I would want ffmpeg on the NAS instead of running it elsewhere? processing 19 tb of video is faster when not done over a network lol

 

[Davvo]

Have you activated the developer mode?

Developer Mode (Unsupported)

Provides information on the unsupported SCALE developer mode and how to enable it.

www.truenas.com

 

[katbyte]

yep sure have

 

[Davvo]

Did you try apt-get install libgl1 or apt-get install ffmpeg libsm6 libxext6?

Another solution might be apt-get install python3-opencv.

Plenty more at the link below.

ImportError: libGL.so.1: cannot open shared object file: No such file or directory

I am trying to run cv2, but when I try to import it, I get the following error: ImportError: libGL.so.1: cannot open shared object file: No such file or directory The suggested solution online is

stackoverflow.com