Kube-Bridge constantly switching states

[Grummeltier]

Hi,
I use TrueNAS-SCALE-22.02.0.1 and I have the following problem. About every 5 Minutes, in my log I see the following output:

Apr 4 15:40:01 truenas kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth64ce1564: link becomes ready Apr 4 15:40:01 truenas kernel: kube-bridge: port 15(veth64ce1564) entered blocking state Apr 4 15:40:01 truenas kernel: kube-bridge: port 15(veth64ce1564) entered disabled state Apr 4 15:40:01 truenas kernel: device veth64ce1564 entered promiscuous mode Apr 4 15:40:01 truenas kernel: kube-bridge: port 15(veth64ce1564) entered blocking state Apr 4 15:40:01 truenas kernel: kube-bridge: port 15(veth64ce1564) entered forwarding state Apr 4 15:40:03 truenas kernel: kube-bridge: port 15(veth64ce1564) entered disabled state Apr 4 15:40:03 truenas kernel: device veth64ce1564 left promiscuous mode Apr 4 15:40:03 truenas kernel: kube-bridge: port 15(veth64ce1564) entered disabled state

I have 4 Truecharts Apps in my TrueNAS and even if I stop all Apps or restart my system, I get those state switches. I don't use IPv6 and my switch don't assign IPv6-Addesses to any of my clients. I have this behaviour a few weeks and unfortunately I can't narrow the time or action down to the Beginning of these log entires. My System runs well and since SCALE 22.02-RC2.

Has anyone an idea how to solve this?

Thanks

 

[truecharts]

File a Jira bug report, that's the best way to go.
This forum is not an iX Systems development forum, so often posts don't even reach a developer at all ;-)

 

[Kroelie]

Hi,
I use TrueNAS-SCALE-22.02.0.1 and I have the following problem. About every 5 Minutes, in my log I see the following output:

Apr 4 15:40:01 truenas kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth64ce1564: link becomes ready Apr 4 15:40:01 truenas kernel: kube-bridge: port 15(veth64ce1564) entered blocking state Apr 4 15:40:01 truenas kernel: kube-bridge: port 15(veth64ce1564) entered disabled state Apr 4 15:40:01 truenas kernel: device veth64ce1564 entered promiscuous mode Apr 4 15:40:01 truenas kernel: kube-bridge: port 15(veth64ce1564) entered blocking state Apr 4 15:40:01 truenas kernel: kube-bridge: port 15(veth64ce1564) entered forwarding state Apr 4 15:40:03 truenas kernel: kube-bridge: port 15(veth64ce1564) entered disabled state Apr 4 15:40:03 truenas kernel: device veth64ce1564 left promiscuous mode Apr 4 15:40:03 truenas kernel: kube-bridge: port 15(veth64ce1564) entered disabled state

I have 4 Truecharts Apps in my TrueNAS and even if I stop all Apps or restart my system, I get those state switches. I don't use IPv6 and my switch don't assign IPv6-Addesses to any of my clients. I have this behaviour a few weeks and unfortunately I can't narrow the time or action down to the Beginning of these log entires. My System runs well and since SCALE 22.02-RC2.

Has anyone an idea how to solve this?

Thanks

Have you had any luck solving this issue?
i've been having this same issue for a while now and i can't seem to solve it.

 

[truecharts]

iX Seems to been working actively on solving some of those issues in either 22.12 or 22.02.3 :)

 

[Grummeltier]

This was caused by my Nextcloud App from Truecharts. I could solved this problem, when I installed the Nextcloud App from the Official Catalog, transfered all my Nextcloud data and than delete the Nexcloud App from the Truecharts Catalog.

 

[TheNarc]

This was caused by my Nextcloud App from Truecharts. I could solved this problem, when I installed the Nextcloud App from the Official Catalog, transfered all my Nextcloud data and than delete the Nexcloud App from the Truecharts Catalog.

+1, this worked for me as well, thanks for pointing me in the right direction, this was driving me crazy for the past month.

 

[DaSnipe]

It's just log spam, but I mean you guys do you lol

 

[truecharts]

+1, this worked for me as well, thanks for pointing me in the right direction, this was driving me crazy for the past month.

The reason you and @Grummeltier was where having these issues, was because we previously (by accident) had a kubernetes service that was not attached correctly to a running process. Sadly enough no-one reported this bug on our bugtracker.

However:
Our completely reworked Nextcloud App (15.x.x) released last week, completely fixed this issue, as well as a bunch of other bugs and performance improvements.

@waqarahmed You might want to take note of the above:
Besides these warnings showing up when an App is disabled, they also show when Apps contain a kubernetes service that is not backed by an open port.

 

[TheNarc]

The reason you and @Grummeltier was where having these issues, was because we previously (by accident) had a kubernetes service that was not attached correctly to a running process. Sadly enough no-one reported this bug on our bugtracker.

However:
Our completely reworked Nextcloud App (15.x.x) released last week, completely fixed this issue, as well as a bunch of other bugs and performance improvements.

@waqarahmed You might want to take note of the above:
Besides these warnings showing up when an App is disabled, they also show when Apps contain a kubernetes service that is not backed by an open port.

I'll try installing Nextcloud again on the weekend, since it's been revamped, thanks for the info.

 

[truecharts]

I'll try installing Nextcloud again on the weekend, since it's been revamped, thanks for the info.

Cool, be sure to send us as much feedback as you can, so we can continue to improve it :)
Hope you like the speedup! ^^

 

[edgecrush3r]

I have been suffering from this same issue. Only way i found around this, was to 'Reset configuration to defaults' and re-import the pool. All configration lost, but at least the system is back online.

 

[TheBuckSergeant]

I have been suffering from this same issue. Only way i found around this, was to 'Reset configuration to defaults' and re-import the pool. All configration lost, but at least the system is back online.

network configurations? i have the same problem all truecharts are up to date still shows kube-bridge:port2 (veth8d01abb1) entered disable state

 

[Hafnernuss]

network configurations? i have the same problem all truecharts are up to date still shows kube-bridge:port2 (veth8d01abb1) entered disable state

Same for me, it appears I have the same problem:

TrueNAS-22.02.3
Nextcloud 24.0.5_15.3.21

 

[truecharts]

Unless people file tickets with our support staff with a complete list of all the Apps they are using, there is not much we can do with this (even though we're referenced quite frequently here).

As we can only guess which app is causing this.

In a TLDR:
The cause is often a kubernets service that is not correctly connecting to an application port.
However: This should still not spam your logs like this, that issue lies with iX and should be reported accordingly.

 

[Sawtaytoes]

Where do I report bugs? I'm not seeing a link here: https://truecharts.org/. Is it only available from the Discord?

I have a bunch of these messages too:

Code:

Dec 24 04:30:02 storeman kernel: device veth35f9286d entered promiscuous mode
Dec 24 04:30:02 storeman kernel: kube-bridge: port 9(veth35f9286d) entered blocking state
Dec 24 04:30:02 storeman kernel: kube-bridge: port 9(veth35f9286d) entered forwarding state
Dec 24 04:30:02 storeman kernel: kube-bridge: port 10(veth7ccd706e) entered blocking state
Dec 24 04:30:02 storeman kernel: kube-bridge: port 10(veth7ccd706e) entered disabled state
Dec 24 04:30:02 storeman kernel: device veth7ccd706e entered promiscuous mode
Dec 24 04:30:02 storeman kernel: kube-bridge: port 10(veth7ccd706e) entered blocking state
Dec 24 04:30:02 storeman kernel: kube-bridge: port 10(veth7ccd706e) entered forwarding state
Dec 24 04:30:03 storeman kernel: kube-bridge: port 6(veth5b6dba9e) entered disabled state
Dec 24 04:30:03 storeman kernel: device veth5b6dba9e left promiscuous mode
Dec 24 04:30:03 storeman kernel: kube-bridge: port 6(veth5b6dba9e) entered disabled state
Dec 24 04:30:07 storeman kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth5331885d: link becomes ready
Dec 24 04:30:07 storeman kernel: kube-bridge: port 6(veth5331885d) entered blocking state
Dec 24 04:30:07 storeman kernel: kube-bridge: port 6(veth5331885d) entered disabled state
Dec 24 04:30:07 storeman kernel: device veth5331885d entered promiscuous mode
Dec 24 04:30:07 storeman kernel: kube-bridge: port 6(veth5331885d) entered blocking state
Dec 24 04:30:07 storeman kernel: kube-bridge: port 6(veth5331885d) entered forwarding state
Dec 24 04:30:12 storeman kernel: kube-bridge: port 6(veth5331885d) entered disabled state
Dec 24 04:30:12 storeman kernel: device veth5331885d left promiscuous mode
Dec 24 04:30:12 storeman kernel: kube-bridge: port 6(veth5331885d) entered disabled state
Dec 24 04:30:14 storeman kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth068c4a11: link becomes ready
Dec 24 04:30:14 storeman kernel: kube-bridge: port 6(veth068c4a11) entered blocking state
Dec 24 04:30:14 storeman kernel: kube-bridge: port 6(veth068c4a11) entered disabled state
Dec 24 04:30:14 storeman kernel: device veth068c4a11 entered promiscuous mode
Dec 24 04:30:14 storeman kernel: kube-bridge: port 6(veth068c4a11) entered blocking state
Dec 24 04:30:14 storeman kernel: kube-bridge: port 6(veth068c4a11) entered forwarding state
Dec 24 04:40:04 storeman kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth46ed7ab7: link becomes ready
Dec 24 04:40:04 storeman kernel: kube-bridge: port 11(veth46ed7ab7) entered blocking state
Dec 24 04:40:04 storeman kernel: kube-bridge: port 11(veth46ed7ab7) entered disabled state
Dec 24 04:40:04 storeman kernel: device veth46ed7ab7 entered promiscuous mode
Dec 24 04:40:04 storeman kernel: kube-bridge: port 11(veth46ed7ab7) entered blocking state
Dec 24 04:40:04 storeman kernel: kube-bridge: port 11(veth46ed7ab7) entered forwarding state

My apps:

• Tailscale
• Plex
• Jellyfin
• Resilio-Sync

All of them are up-to-date, so one of them is probably having the failure to connect issue.

 

[GrimmReaperNL]

Did this get fixed? I'm on TrueNAS-SCALE-22.12.0 and am not using NextCloud. My console has a constant spam of this.

 

[Sawtaytoes]

No. None of my apps have been fixed, and I've added more which made this issue worse.

 

[3133computing]

I have the same issue. Additionally, after a few days of log spam, the network goes down (disabled state) and doesn't come back unless I restart the machine.

 

[smartypantsuk]

Same issue here. Strange thing is I'm seeing multiple devices as though there is more than one interface, unless I'm misunderstanding it.

Dec 24 04:30:02 storeman kernel: device veth35f9286d entered promiscuous mode
Dec 24 04:30:02 storeman kernel: kube-bridge: port 9(veth35f9286d) entered blocking state
Dec 24 04:30:02 storeman kernel: kube-bridge: port 9(veth35f9286d) entered forwarding state
Dec 24 04:30:02 storeman kernel: kube-bridge: port 10(veth7ccd706e) entered blocking state
Dec 24 04:30:02 storeman kernel: kube-bridge: port 10(veth7ccd706e) entered disabled state
Dec 24 04:30:02 storeman kernel: device veth7ccd706e entered promiscuous mode
Dec 24 04:30:02 storeman kernel: kube-bridge: port 10(veth7ccd706e) entered blocking state
Dec 24 04:30:02 storeman kernel: kube-bridge: port 10(veth7ccd706e) entered forwarding state
Dec 24 04:30:03 storeman kernel: kube-bridge: port 6(veth5b6dba9e) entered disabled state
Dec 24 04:30:03 storeman kernel: device veth5b6dba9e left promiscuous mode
Dec 24 04:30:03 storeman kernel: kube-bridge: port 6(veth5b6dba9e) entered disabled state
Dec 24 04:30:07 storeman kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth5331885d: link becomes ready
Dec 24 04:30:07 storeman kernel: kube-bridge: port 6(veth5331885d) entered blocking state
Dec 24 04:30:07 storeman kernel: kube-bridge: port 6(veth5331885d) entered disabled state
Dec 24 04:30:07 storeman kernel: device veth5331885d entered promiscuous mode

No nextcloud here, although it was installed at one point but I took it off as it never worked.

 

[Valvator]

I have been suffering from this same issue. Only way i found around this, was to 'Reset configuration to defaults' and re-import the pool. All configration lost, but at least the system is back online.

It also worked for me.