After upgrade from 11.3 u3 to 12 - no network connectivity, most jails do not start

ci4ic4

Cadet
Joined
Jul 1, 2020
Messages
5
I have been running FreeNAS 11 for at least 18 months under XCP-NG without any problems. Today, after I saw the message about the release of TrueNAS-CORE Beta1, I decided to try it. The upgrade completed without any problems, the pools were available as before, the network configuration was retained, but I could not send or receive any packets on the xn0 and xn1 interfaces. I tried resetting them, manually and using netcli, to no avail; just in case I also swapped the networks they were attached to, thinking there might be difference in enumeration them. Nothing would work, unfortunately.

Somewhat academical, after the lack of networking; two out of the three jails which should have started automatically did not start (nextcloud and emby - with a strange message about a missing fifth element or similar, did not take the exact message). The third - wiki - started ok. I didn't have any time to investigate thi problem.

Ofcourse I was able to activate the old boot environment and carry on using 11.3, so for now all is well.

FYI I also tried a new installation of TrueNAS-CORE on the same XCP-NG host, this time all was well and the networking operated as expected, so it appears it is a problem related to the update itself.

I know well that running FreeNAS as a virtual machine is not recommended, but both the performance and the reliability so far have been quite remarkable for me and forthe moment I do not have another option. The other FreeBSD 12 VMs I have on the same host all have no problems as well.
 

ThreeDee

Guru
Joined
Jun 13, 2013
Messages
698
I had a similar experience updating to 12 Beta and I'm not running FreeNAS in a VM .. I have a simple setup with just 2 jails, Plex and UniFi controller .. neither one would start. I tried to "Update" the Unifi controller via the jail "Update" button .. but it failed. Booted back into 11.3 and back to working jails , etc ..
 

Yorick

Wizard
Joined
Nov 4, 2018
Messages
1,912
Thanks for testing! So the rest of us don’t have to go through that experience: Try the upgrade again, go to system support and submit a ticket with a debug? That way, ix has a fighting chance of fixing whatever it is that caused you to lose networking.
 

Alecmascot

Guru
Joined
Mar 18, 2014
Messages
1,175
From the release notes :

"Installed plugins also require an upgrade to function with TrueNAS 12.0-Beta1. However, this upgrade is one-way; reverting the system to an earlier version of TrueNAS means any upgraded plugins will cease to function. If you wish to irreversibly upgrade your plugins after updating TrueNAS, open the shell and enter iocage upgrade <plugin-jail-name>. For example, to upgrade a weechat plugin that was named “weechattest”, enter iocage upgrade weechattest.
 

Yorick

Wizard
Joined
Nov 4, 2018
Messages
1,912
For some value of “require”. All my 11.3 base jails are working just fine, and I’m in no hurry to bring them to 12.1. Maybe that depends on what’s running in the jail, to an extent.

Agreed that upgrading jails is a step one would take once fully committed.

I wonder whether plugins would differ. They are “just” jails with a bit of a config wrapper, after all.
 

ci4ic4

Cadet
Joined
Jul 1, 2020
Messages
5
The problem with the plugins was secondary in my case. The main problem was the total loss of network connectivity after the upgrade, no packets coming in or going out whatsoever, whereas the fresh installation worked ok, so there should be no issue with the xn driver or elsewhere. I am more interested if somewhen else testing the upgrade has seen similar.
 

ByteNick

Explorer
Joined
Jan 24, 2015
Messages
98
Same issue here, updating from 11.3-U5 to 12.
Kills the network connectivity as soon as it starts the jails.

Any idea why?
 

inis

Dabbler
Joined
Aug 9, 2017
Messages
23
Same issue here, updating from 11.3-U5 to 12.
Kills the network connectivity as soon as it starts the jails.

Any idea why?

Change the devfs_ruleset under jail properties worked for me on all but 2, still working on it as I type but might get you heading in the right direction.
 

inis

Dabbler
Joined
Aug 9, 2017
Messages
23
Change the devfs_ruleset under jail properties worked for me on all but 2, still working on it as I type but might get you heading in the right direction.


So plex, radarr, sonarr and sab I changed to 1-4 , nextcloud and unifi i changed to 1001 and they came up. Hope this helps.
 

x130844

Dabbler
Joined
Oct 25, 2020
Messages
46
Same issue here, updating from 11.3-U5 to 12.
Kills the network connectivity as soon as it starts the jails.

Any idea why?
for me, it was the auto NIC , on freenas it used to work, now it doesn't on truenas.
 

ByteNick

Explorer
Joined
Jan 24, 2015
Messages
98
It's getting worse:
My setup is a Unifi Dreammachine Pro, plus some Unifi Swtches and AP's. Freenas up until 11.3 was running perfectly, with many, many jails installed.

After the upgrade to TruesNAS 12, the system boots, but when it loads the jails, cuts the entire network connectivity through our the entire LAN.

I really don't know what to do. Also, @x130844, what do you mean by auto NIC?
 

x130844

Dabbler
Joined
Oct 25, 2020
Messages
46
It's getting worse:
My setup is a Unifi Dreammachine Pro, plus some Unifi Swtches and AP's. Freenas up until 11.3 was running perfectly, with many, many jails installed.

After the upgrade to TruesNAS 12, the system boots, but when it loads the jails, cuts the entire network connectivity through our the entire LAN.

I really don't know what to do. Also, @x130844, what do you mean by auto NIC?
When you let truenas use the Nic, set as auto. instead of you, specifying which NIC port.
In my case, auto would create issues. I had to manually tell truenas to use my cxl2 or whatever I wanted to use. (cxl0 and cxl1 weren't connected, it should have picked up the next eth port.)
 

Whe3zy

Dabbler
Joined
Sep 4, 2020
Messages
21
Ive had a few issues as well with the upgrade and things that worked in v11 all of a sudden do not work in v12. I'm debating if its worth re-imaging the OS drive from scratch in hopes that my RaidZ storage pool and iocage pool on the other HDD function like they should again upon rejoining them to a fresh image????
ksnip_20201113-185244.png
 
Last edited:

ByteNick

Explorer
Joined
Jan 24, 2015
Messages
98
Problem solved.
Although perfectly fine under FreeNas 11.x, the upgrade to TrueNAS 12 revealed the fact that having 2 NIC's on the same subnet was killing the entire LAN connectivity. I found this by reading the docs, a rather easy thing to overlook.
 

Dwayneson

Dabbler
Joined
Jun 3, 2016
Messages
24
Problem solved.
Although perfectly fine under FreeNas 11.x, the upgrade to TrueNAS 12 revealed the fact that having 2 NIC's on the same subnet was killing the entire LAN connectivity. I found this by reading the docs, a rather easy thing to overlook.
Hi ByteNick,
I think I'm having the exact same problem. Upgraded to 12 and now boot freezes on Updating CPU Micronode.
Can you please point me where to look to help solve this? I'm a definite newb so anything you can add will help massively.
Thank you.
 

ByteNick

Explorer
Joined
Jan 24, 2015
Messages
98
Hi ByteNick,
I think I'm having the exact same problem. Upgraded to 12 and now boot freezes on Updating CPU Micronode.
Can you please point me where to look to help solve this? I'm a definite newb so anything you can add will help massively.
Thank you.
A very, very quick way is to physically unplug the second network cable linking you TrueNAS mahine to the router.
 
Joined
Dec 30, 2014
Messages
6
Problem solved.
Although perfectly fine under FreeNas 11.x, the upgrade to TrueNAS 12 revealed the fact that having 2 NIC's on the same subnet was killing the entire LAN connectivity. I found this by reading the docs, a rather easy thing to overlook.

I wanted to reiterate ByteNick's comment. I assumed this only meant 2 active NIC's but even with my old NIC removed (but the interface still listed), I still ran into the issue of no network in my jails/iocages. It didn't matter what combination of settings I tried (VNET, BPF, auto, static, dhcp, hardware offload, etc.), it would not work until I deleted the old interface. Technically NAT worked but that's not an ideal solution for most. Hopefully, the next person googling for this error sees this.
 
Top