linux-image-truenas-amd64 is unable to finish installing

[rudydelorenzo]

Hi everyone!

I'm running TrueNAS SCALE and I did an apt-get upgrade, which upgraded a couple packages, including linux-image-truenas-amd64

However it encountered an error and now the package is "not fully installed".

Running

Code:

dpkg --configure -a

yields the following:

Code:

root@truenas[~]# dpkg --configure -a                               
Setting up linux-image-truenas-amd64 (5.10.81+truenas-1) ...
/etc/kernel/postinst.d/dkms:
dkms: running auto installation service for kernel 5.10.81+truenas:Error!  The /var/lib/dkms/wireguard/1.0.20210219/5.10.81+truenas/x86_64/dkms.conf for module wireguard includes a BUILD_EXCLUSIVE directive which
does not match this kernel/arch.  This indicates that it should not be built.
.
/etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-5.10.81+truenas
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
grep: /etc/initramfs-tools/modules: No such file or directory
cryptsetup: ERROR: Couldn't resolve device boot-pool/ROOT/22.02-RC.2
cryptsetup: WARNING: Couldn't determine root device
cryptsetup: WARNING: target 'md127' not found in /etc/crypttab
I: The initramfs will attempt to resume from /dev/dm-0
I: (/dev/mapper/md127)
I: Set the RESUME variable to override this.
/etc/kernel/postinst.d/kdump-tools:
kdump-tools: Generating /var/lib/kdump/initrd.img-5.10.81+truenas
mkinitramfs: failed to determine device for /
mkinitramfs: workaround is MODULES=most, check:
grep -r MODULES /var/lib/kdump/initramfs-tools

Error please report bug on initramfs-tools
Include the output of 'mount' and 'cat /proc/mounts'
update-initramfs: failed for /var/lib/kdump/initrd.img-5.10.81+truenas with 1.
run-parts: /etc/kernel/postinst.d/kdump-tools exited with return code 1
dpkg: error processing package linux-image-truenas-amd64 (--configure):
 installed linux-image-truenas-amd64 package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 linux-image-truenas-amd64

Hope someone can help me out! Thanks!

 

[jgreco]

TrueNAS is an appliance firmware, not a general purpose Linux or FreeBSD OS for you to be tinkering with like this.

Reinstall from ISO. For future upgrades, use the GUI.

 

[disklord]

I ran into the same issue... seriously we have to clean install to fix this?? Surely TrueNAS a simple "apt update" in the terminal should break your entire install. That is a major oversight if so

 

[jgreco]

I ran into the same issue... seriously we have to clean install to fix this?? Surely TrueNAS a simple "apt update" in the terminal should break your entire install. That is a major oversight if so

Surely a simple "rm -fr /" in the terminal window shouldn't break your entire install.

I just explained in the message immediately preceding yours that TrueNAS is not a general purpose Linux OS. If you run commands from the CLI that iXsystems did not tell you to run, it's pretty rude of you to consider this

a major oversight

rather than a PEBCAK issue. The ONLY mechanism iXsystem provides to modify the base system is the installer/updater. So unless some iX employee or developer gives you a set of things to do to your TrueNAS SCALE install, please don't act all shocked or offended when you try some random Linux commands you use on other systems and it doesn't work the way you expect.

iXsystems employees are badged on these forums with "iXsystems" under their avatar.

iXsystems developers are also tagged in the Jira.

 

[disklord]

Surely a simple "rm -fr /" in the terminal window shouldn't break your entire install.

I just explained in the message immediately preceding yours that TrueNAS is not a general purpose Linux OS. If you run commands from the CLI that iXsystems did not tell you to run, it's pretty rude of you to consider this



rather than a PEBCAK issue. The ONLY mechanism iXsystem provides to modify the base system is the installer/updater. So unless some iX employee or developer gives you a set of things to do to your TrueNAS SCALE install, please don't act all shocked or offended when you try some random Linux commands you use on other systems and it doesn't work the way you expect.

iXsystems employees are badged on these forums with "iXsystems" under their avatar.

iXsystems developers are also tagged in the Jira.

Please, cut the nonsense for a second. I am not going around running "rm" on my system here like some kind of blithering idiot that you make me out to be. This is not some "random" linux command. It's a standard Debian command to update packages, which has been used in EVERY Debian based distro, ever.

If iXSystems has decided to butcher Debian in a way that running a standard package manager update breaks your system, it is your obligation to warn users about this. This is the initial message that you get when logging in via SSH - you warn me about settings in the CLI not being permenant, but there is NO mention of anything else: https://i.imgur.com/sOqVPMz.jpeg

I'd recommend adding a warning for anyone else who might get the impression that FreeNAS Scale runs on standard Debian.

 

[jgreco]

It's a standard Debian command to update packages, which has been used in EVERY Debian based distro, ever.

Well, there's your problem. This isn't a "Debian based distro." It is an appliance. It is provided as a binary blob. It is not there for your hacking. FreeNAS and now TrueNAS have always been provided as a monolithic and highly customized version of the operating systems they are based on. They are not just a package that you add to a generic "Debian based distro" or FreeBSD system.

If iXSystems has decided to butcher Debian in a way that running a standard package manager update breaks your system, it is your obligation to warn users about this.

Well, that doesn't seem to be what's happened. It seems more like you assumed that TrueNAS was just Debian, when, in fact, you were never told such a thing.

I'd recommend adding a warning for anyone else who might get the impression that FreeNAS Scale runs on standard Debian.

So where does that nanny-ism start and end? You happened to think it was a Debian OS you could hack on using standard Debian stuff. What happens when the next guy recognizes it as a "generic Linux of some sort" and tries to update the kernel?

What happens when the third guy recognizes it as "some sort of UNIX-ish system" and tries generic UNIX stuff?

No one told any of you that any of this was actually acceptable. It's an appliance.

Please, cut the nonsense for a second.

Yes, indeed, please cut the nonsense. The marvelous bit of this open source project is that it isn't all entirely locked up behind a wall of impenetrable GUI and services. But this is also the risk. You made a mistaken assumption and you seem unwilling to accept the consequences.

I have spent more than a decade on these forums trying to guide people to the right path, but also happily discussing the wrong path with anyone willing to own the results of their actions. Don't want to use ECC? I'll explain the risks and why it's a problem, then let you to it. Don't want to use sync writes? I'll explain the risk factors. Want to undersize your RAM? Use RAIDZ for block storage? Use a non-recommended RAID controller? I, and other people on these forums, are happy to discuss all of this with you. But you have to be willing to accept the consequences of your actions.

Made a mistake and mistook this for a generic Debian distro? No amount of complaining about it is going to transform TrueNAS into a generic Debian distro, so the thing to do is to figure out where to go from here. My suggestion: Graciously accept the correction I'm offering you, and then move on. It's fine. It's totally recoverable. Download your config. reinstall the OS, upload your config, and get on with life. It's a potentially understandable error. We correct mistaken users all the time, many of whom go on to become remarkable contributing members of the community. We don't think less of new users for having made mistakes.

 

[disklord]

Okay, I'll admit I messed up. I only read the Introduction to TrueNAS Scale thinking that would be enough to get started, but it looks like a warning is present in the release notes. At least this post now serves as a heads up for anyone else who runs into this problem.

Either way, I don't think what I am suggesting is entirely unreasonable - I have used many Linux-based appliances (an appliance is merely a type of distro, by the way) in the past, and either they don't have these system-breaking issues with the inbuilt package manager, or they provide warnings in the login welcome (MOTD) message. All I am asking is that the welcome message be updated to give users a quick heads up, just in case they make the same reasonable assumption that I did. I can assure you this is not "nanny" behaviour. It will save a lot of frustration.

The good news is that the inbuilt config export functionality is mostly working, and exporting and re-importing configs onto a clean install of TrueNAS Scale was a lot easier than I expected, so I am at least grateful for that.

 

[Ericloewe]

I don't remember for sure what iX said about how this was going to be solved, when the issue was brought up early on, but I seem to recall something about clearing out the configured repositories to keep apt from getting "random" stuff.

 

[jgreco]

Either way, I don't think what I am suggesting is entirely unreasonable - I have used many Linux-based appliances (an appliance is merely a type of distro, by the way) in the past, and either they don't have these system-breaking issues with the inbuilt package manager, or they provide warnings in the login welcome (MOTD) message.

TrueNAS is a well-established system, and over the last decade, we ran into this sort of issue repeatedly with FreeBSD. iXsystems developers have made it clear that they're not actually interested in trying to determine how to break the system (such as, let's say, removing the Linux "apt" or FreeBSD "pkg") to prevent this, because there are "future plans" or because they are "used internally" or a few other things we've been told. They also didn't want to provide warnings in the MOTD, or threatening logging messages like ESXi, or any of the other things suggested in the last ~hundred threads of this nature. Since none of us present in this thread have the ability to force iXsystems to change course, the only resolution that is vaguely satisfying is to educate on what the design is, and get to this point:

warning is present in the release notes. At least this post now serves as a heads up for anyone else who runs into this problem.

As do all the other posts over the years. You can probably just pull up keyword "appliance" in the search box and find hundreds of threads about not modifying the system. Unfortunately, the problem is that people don't actually search for these or read the release notes, certainly not before getting into trouble, so that puts those of us who are trying to help provide community support in a bad position of having to repeatedly explain this, over and over, year after year.

All I am asking is that the welcome message be updated to give users a quick heads up, just in case they make the same reasonable assumption that I did. I can assure you this is not "nanny" behaviour. It will save a lot of frustration.

I'm not the person you need to convince. You have just as much ability to effect this change as I do. The people who need to be convinced have been ... the polite word is "resistant", I think.

 

[anodos]

I don't remember for sure what iX said about how this was going to be solved, when the issue was brought up early on, but I seem to recall something about clearing out the configured repositories to keep apt from getting "random" stuff.

Currently in nightlies apt binaries do not have execute bit set on them to prevent users from using them.

Code:

root@truenas[~]# apt update
zsh: permission denied: apt

This will hopefully avoid some of these problems, but leave open the option for some tinkerers to chmod the files and do what they want to (with the knowledge that they shouldn't file bug tickets in this case :) )

 

[Tolki]

`apt` is necessary for things like HFS+ access, which I used just today to backup an old drive.

The root problem, *ie* the fact that `apt upgrade` specifically breaks the package manager, should be addressed. There is no good reason for it.

 

[jgreco]

There is no good reason for it.

There is indeed a good reason for it; NONE of us, whether the people at iXsystems developing the product, or those of us here in the community trying to provide support for the product, want community members making random changes to the basic system, running into newly introduced problems that ONLY exist for that one user in the world who has apt-installed some random thingy, and then the user shows up here in the forums or filing Jira tickets for some effectively unreproducible issue caused by the user's offroad adventures. TrueNAS is an appliance and does not use the Debian command line package manager. Adding packages to the base system is unsupported and the developers do not want that support nightmare.

Now, you have previously been rude in this thread, which was deleted by one of the moderators, and the history/issues/status have been explained by me, and even one of the developers has provided a little additional feedback. It is clear that you do not agree with this design decision, but it is not within your power or mine to force iXsystems to do things the way you would prefer.

As @anodos has implied, you are free to "fix" things in whatever manner you wish, but in such case, please don't file Jira tickets and also please do not file complaints in the forums about any offroading misadventures you have.

There is no reason to believe that any amount of debate here is going to make this work the way you would prefer. Part of the job of the moderators here is to keep the discussion on track and to provide clarity and information. It is incumbent on the users to listen even when they are not hearing the things they'd like to. With that said, I'm closing this thread as it seems that leaving it open is just encouraging further argument. If you have something constructive and productive to add, please message me and I'm happy to reconsider.

 

[Ericloewe]

Just a note for prosperity, and proof that civilized feedback is welcome and useful: The MOTD has been updated to more emphatically warn about this (both in Scale and Core), see https://github.com/truenas/middleware/pull/8322