How to edit GRUB_CMDLINE_LINUX_DEFAULT?

gator96100

Cadet
Joined
Sep 11, 2021
Messages
2
I was running Proxmox on my home server and now I am trying to switch to TrueNAS Scale. I did run into a problem with passing devices into the VM.
[EFAULT] internal error: qemu unexpectedly closed the monitor: 2021-09-11T14:34:10.839083Z qemu-system-x86_64: -device vfio-pci,host=0000:04:00.0,id=hostdev0,bus=pci.0,addr=0x7: vfio 0000:04:00.0: group 13 is not viable Please ensure all devices within the iommu_group are bound to their vfio bus driver.

I did remember I had the same issue on Proxmox, but I was able to separate the IOMMU groups by adding “pcie_acs_override=downstream” to the kernel boot commandline.
I tried to edit “/etc/default/grub.d/truenas.cfg” and ran update-grub afterwards, but after a reboot the file was overwritten. What is the correct way of modifying the grub commandline? Also it would by nice to have a GUI option for ACS override.
 

tuky

Cadet
Joined
Jan 19, 2022
Messages
4
I would also like to learn about this, because I can successfully use the amdgpu driver instead of the radeon driver by adding these options to the linux command line in GRUB: https://wiki.debian.org/AtiHowTo#AMDGPU.2FVulkan_on_older_cards

radeon.cik_support=0 amdgpu.cik_support=1 radeon.si_support=0 amdgpu.si_support=1 radeon.dpm=0

however that is temporary and I would like to persist these options.
 

ksimm1

Dabbler
Joined
Dec 7, 2020
Messages
42
I did remember I had the same issue on Proxmox, but I was able to separate the IOMMU groups by adding “pcie_acs_override=downstream” to the kernel boot commandline.
I tried to edit “/etc/default/grub.d/truenas.cfg” and ran update-grub afterwards, but after a reboot the file was overwritten. What is the correct way of modifying the grub commandline? Also it would by nice to have a GUI option for ACS override.

The below command will append the kernel options you define to the standard truenas GRUB options and will persist across reboots
Code:
midclt call system.advanced.update '{"kernel_extra_options": "firstoption=value secondoption=value thirdoption=value"}'
 

tuky

Cadet
Joined
Jan 19, 2022
Messages
4
Thank you so much. This seems to be a great new, yet undocumented feature. Will try it and report back.
 

user623

Dabbler
Joined
Jan 20, 2021
Messages
18
May be add this request in System-Advanced ? On CORE many kernel options controlled over loader.conf+sysctl. On SCALE sysctl work, but kernel parameters not accessible.
 

theippenguin

Cadet
Joined
Aug 1, 2022
Messages
2
The below command will append the kernel options you define to the standard truenas GRUB options and will persist across reboots
Code:
midclt call system.advanced.update '{"kernel_extra_options": "firstoption=value secondoption=value thirdoption=value"}'
Tried running this comand in TrueNAS-SCALE-22.02.2.1 and only recieve "command not found:". I'm no linux expert and new to truenas but it seems it is not added to the path as everything I have seen in forums looks like it has been added to Truenas. Can you tell me what path this command is at?
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,398
Tried running this comand in TrueNAS-SCALE-22.02.2.1 and only recieve "command not found:". I'm no linux expert and new to truenas but it seems it is not added to the path as everything I have seen in forums looks like it has been added to Truenas. Can you tell me what path this command is at?
Are you running as root or as a regular user in a SSH session? The command is in root's PATH.
 

skittlebrau

Explorer
Joined
Sep 1, 2017
Messages
53
I was running Proxmox on my home server and now I am trying to switch to TrueNAS Scale. I did run into a problem with passing devices into the VM.


I did remember I had the same issue on Proxmox, but I was able to separate the IOMMU groups by adding “pcie_acs_override=downstream” to the kernel boot commandline.
I tried to edit “/etc/default/grub.d/truenas.cfg” and ran update-grub afterwards, but after a reboot the file was overwritten. What is the correct way of modifying the grub commandline? Also it would by nice to have a GUI option for ACS override.
Can anyone confirm whether the ACS override patch was compiled into TrueNAS SCALE's kernel?

I run TrueNAS SCALE baremetal and I've added pcie_acs_override=downstream,multifunction into GRUB/kernel options with the midclt command above and I can confirm it's in there and persistent across reboots, but my IOMMU groups are still very restrictive as if the ACS patch has no effect. I've tested in Proxmox with the same ACS override patch in GRUB and all my IOMMU groups are split up like they should be.
 

Kosygor

Dabbler
Joined
Sep 28, 2021
Messages
18
I second this. I have added [B][B]pcie_acs_override=downstream[/B][/B] and groups did not split.
 

m9x3mos

Dabbler
Joined
May 13, 2021
Messages
34
Has anyone tried with the latest release?
I think I saw something about kernel updates and not sure if that made a difference.
Currently I am running it under Windows as the hypervisor and iscsi as the storage target for my VMs but the storage speed is faster in native VMs when truenas is running baremetal.
 
Top