TrueNAS 13.0-U2 is now available

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,691
So I've two screenshots. First one is right before the panic, second is it.
True-NAS-13-0-U2-before-panic.png

True-NAS-13-0-U2-panic.png

Please "report-a-bug".... especially if its replicable.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
So I've two screenshots. First one is right before the panic, second is it.
True-NAS-13-0-U2-before-panic.png

True-NAS-13-0-U2-panic.png

OK, the panic happened in core_pcpu_init() when trying to load the kernel CPU initialization routine. Typically a problem here is due to the hypervisor reporting the wrong number of cores to TrueNAS. How many cores have you allocated to TrueNAS in ProxMox? How many physical cores do you have in your Ryzen CPU?
 

jlpellet

Patron
Joined
Mar 21, 2012
Messages
286
Hi,

just updated from TrueNAS 13.0-U1.1. Sadly U2 doesn't come up completely, I had to boot into U1.1 environment again. I tried twice, but U2 always makes the kernel panic.
Unfortunately I just don't know how to provide details. How can I help here?

Environment: TrueNAS 13.0-U1.1 in KVM on Proxmox. I'm using this environment since 12.0-U7 and updated always went smooth.

Bye,
Chris
No obvious answers but would try the following:
1) do a manual update rather than in-line update
2) ensure no more than 3 boot environments before the update
3) scrub the boot pool before updating
Good luck.
John
 

Frosty0571

Cadet
Joined
Aug 31, 2022
Messages
2
Hi,
I have the same issue on Proxmox 6.4-13, running the update from TrueNAS-13.0-U1.1.
Getting the same kernel panic and switching back to U1.1 is working great.
CPU is configured as "2 (1 sockets, 2 cores) [host] [cpuunits=1000]".
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
Hi,
I have the same issue on Proxmox 6.4-13, running the update from TrueNAS-13.0-U1.1.
Getting the same kernel panic and switching back to U1.1 is working great.
CPU is configured as "2 (1 sockets, 2 cores) [host] [cpuunits=1000]".

core_pcpu_init() changed in upstream FreeBSD. Try configuring your CPU as 2 (2 sockets, 1 core).
 

Frosty0571

Cadet
Joined
Aug 31, 2022
Messages
2
core_pcpu_init() changed in upstream FreeBSD. Try configuring your CPU as 2 (2 sockets, 1 core).
Thanks Samiel, I'll try!
Do you have any reference for the change of `core_pcpu_init()` in the FreeBSD code? Would be good to know.
Any link in the release notes of U2 would also be great.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399

adrianwi

Guru
Joined
Oct 15, 2013
Messages
1,231
I've upgraded from U1 to U2 without any issues. Thanks to everyone involved!
 

TrueChris

Cadet
Joined
Dec 12, 2021
Messages
8
OK, the panic happened in core_pcpu_init() when trying to load the kernel CPU initialization routine. Typically a problem here is due to the hypervisor reporting the wrong number of cores to TrueNAS. How many cores have you allocated to TrueNAS in ProxMox? How many physical cores do you have in your Ryzen CPU?

To answer all questions:
1. I didn't upgrade the pool since installing 13.0
2. I did a successful scrub.
3. The system has 24GB of RAM.
4. The system doesn't use a thumb drive but a virtual 10GB harddisk.

The TrueNAS VM gets two cores in one socket from Proxmox. The host CPU is a Xeon E5-2630 v2 (12 core). Virtualized CPU type is equal real CPU.

In the meantime I trimmed the boot environment to one (U1.1) before the update and switched from above CPU configuration to 2 sockets, 1 cores and also 1 sockets, 1 cores but behaviour stayed the same.

I don't use any Jails or VM inside TrueNAS.

So I just created https://ixsystems.atlassian.net/browse/NAS-117975

Regards,
Chris
 

pkerwien

Cadet
Joined
Aug 31, 2022
Messages
3
To answer all questions:
1. I didn't upgrade the pool since installing 13.0
2. I did a successful scrub.
3. The system has 24GB of RAM.
4. The system doesn't use a thumb drive but a virtual 10GB harddisk.

The TrueNAS VM gets two cores in one socket from Proxmox. The host CPU is a Xeon E5-2630 v2 (12 core). Virtualized CPU type is equal real CPU.

In the meantime I trimmed the boot environment to one (U1.1) before the update and switched from above CPU configuration to 2 sockets, 1 cores and also 1 sockets, 1 cores but behaviour stayed the same.

I don't use any Jails or VM inside TrueNAS.

So I just created https://ixsystems.atlassian.net/browse/NAS-117975

Regards,
Chris
I have a very similar setup but with Intel Xeon CPU E5-2620 v4 CPU, 4 vCPUs, 16GB RAM for the TrueNAS VM and saw exactly the same problem when I tried to update from 13.0-U1.1 to -U2 yesterday.
 

Lefuneste

Cadet
Joined
Dec 7, 2018
Messages
5
Same issue here on Proxmox 7.2-7 on Intel i7 8800K CPU hypervisor. KDB panic after upgrading to U2. Changing the CPU model from HOST to KVM64 allowed the system to boot properly.
 

mav@

iXsystems
iXsystems
Joined
Sep 29, 2011
Messages
1,428
core_pcpu_init() function is a part of hwpmc(4) CPU profiling subsystem. It seems Proxmox virtualizes most of the PMC hardware, but not
MSR_DEBUGCTLMSR CPU register, that got used in 13.0-U2 to get more accurate profiles. It can be that switching CPU model you change emulation of that register. I've tested my work on VMware and had no problems there. But just to be safe I think I'll just block that new functionality when inside VMs. Any way profiling inside VM is not the best idea, it can't be very precise.

PS: Should be patched in next nightly build. Confirmation would be good.
 
Last edited:

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
Upgraded from 13.1-U1.1 and everything is going great. Original install was 9.3 and I have upgraded ever since :) same usb key from 2015.
 
Joined
May 2, 2017
Messages
211
Upgraded from 13.0-RELEASE to U2 and now none of my jails have DNS resolution. I ping google.com and I get an error that says "cannot resolve", and "host name lookup failure". Ughhhh.... TrueNAS shell can ping. Jail shells cannot.
 
Joined
May 2, 2017
Messages
211
Multiple reboots not solving the DNS issue. Boot back into 13.0-RELEASE and everything works again.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
How is your jail networking set up?
 
Joined
May 2, 2017
Messages
211
How is your jail networking set up?
VNET and Berkely Packet Filter are enabled. VNET interface to auto, manually configured 172.16.2.0/24 address with default route of 172.16.2.1 and no IPv6. Network Properties with vnet0:bridge0 and ip4.saddrsel and ip6.saddrsel selected. Also checked are allow_set_hostname, allow_raw_sockets and allow_mlock.

It's been like this forever. Nothing different except for the update to TrueNAS.

All the jails run 12.2-RELEASE-p13 except one on 12.3-RELEASE.
 

SweetAndLow

Sweet'NASty
Joined
Nov 6, 2013
Messages
6,421
VNET and Berkely Packet Filter are enabled. VNET interface to auto, manually configured 172.16.2.0/24 address with default route of 172.16.2.1 and no IPv6. Network Properties with vnet0:bridge0 and ip4.saddrsel and ip6.saddrsel selected. Also checked are allow_set_hostname, allow_raw_sockets and allow_mlock.

It's been like this forever. Nothing different except for the update to TrueNAS.

All the jails run 12.2-RELEASE-p13 except one on 12.3-RELEASE.
1. update your jails
2. could it be Realtek RTL8111H networking?
 
Top