kube-bridge killing network. Need help

[SimoneF]

Current network setup:
Eno1 interface without ip set
Br01 interface created via GUI with static IP set, eno1 member of br01.
This was necessary to share an nfs share from the host to VM.
(Naive question: why isn't this the default setting?)

After boot for the first seconds the webui Is accessible, but After "kube-bridge" loads creating other virtual network interfaces (I think), the WebUI Is not reachable anymore.

I tried disabling kube-router and docker services without success.

Since Kubernetes integration Is not yet stable how can I deactivate everything about it so that, at least, the WebUI can be reachable?

Thanks

 

[morganL]

Seems like the networking design is too complex for one IP address.... TrueNAS + VM + Kubernetes. I'd suggest describing all the service requirements and reviewing the IP addressing model.

 

[SimoneF]

Seems like the networking design is too complex for one IP address.... TrueNAS + VM + Kubernetes. I'd suggest describing all the service requirements and reviewing the IP addressing model.

It's not only one ip address, every VM connected to the bridge has its own ip address.
This Is the first time I use truenas scale, I've always used proxmox so in some things I'm a bit lost.
The configuration I used in proxmox Is the default One, every virtual interface of every VM connected to the host bridge, ip addresses set statically in every vm or given with dhcp by the external router.
This is what I tried to replicate in TrueNas SCALE using the GUI, but the result is the One described, when kube-bridge starts it screw everything, and I've never used Kubernetes or docker and the automation that happens at boot time to me it's a black box and I don't know where to put my hands on. (To deactivate It and avoid It to start at boot time)

Sorry if my english might seem confusing or with errore but it's not my native language.

Thanks again,
Simone

 

[morganL]

I'd suggest trying an IP address on your primary ethernet.... have VM access NFS share via that IP. If that's not working, we can try to work out why....

 

[SimoneF]

I'd suggest trying an IP address on your primary ethernet.... have VM access NFS share via that IP. If that's not working, we can try to work out why....

If you mean to not create the bridge and use the primary interface, nfs share of course would not work, as expected. To have the the host visible to the kvm guest you need a bridge interface.

Creating the bridge will let everything work, as expected. But at the first reboot, when kube-bridge loads, problems will appear.

 

[KrisBee]

Since Kubernetes integration Is not yet stable how can I deactivate everything about it so that, at least, the WebUI can be reachable?
Thanks

@SimoneF Once you've triggered the background processes to setup Kubermetes ( k3s, kube router, ix-system datasets on the data pool, etc) AFAIK there's no simple undo. Does a systemctl restart ix-netif.service make the WebUI accessible?

 

[SimoneF]

@SimoneF Once you've triggered the background processes to setup Kubermetes ( k3s, kube router, ix-system datasets on the data pool, etc) AFAIK there's no simple undo. Does a systemctl restart ix-netif.service make the WebUI accessible?

No luck.
I might use another pcie eth card to solve this problem until this bug has been fixed.
Or maybe reinstall SCALE and don't select a pool for Applications so that Kubernetes won't initialize at all (hopefully)
Do you know if on nightly releases this bug Is present?

 

[KrisBee]

@SimoneF I've only looked at 20.12 ALPHA running in a VM with nested virtualisation and I don't see the problem you have when the kubernetes set-up is invovked. Although my VM was not set start on boot

k3s etc won't initialise unless you start using the APPS or is manually invoked at the CLI. Looks like you'll have to re-install SCALE.

Networking is temperamental. I started with DHCP on the NIC as per default setup, then remove IP and created a bridge with static IP and nic as a member BEFORE attempting to save he new config. When it goes into the "test" state, open a new browser tab to login on the static IP and save the new network config. Or you can setup a static IP at the console, immediately after installing SCALE.

 

[SimoneF]

@SimoneF I've only looked at 20.12 ALPHA running in a VM with nested virtualisation and I don't see the problem you have when the kubernetes set-up is invovked. Although my VM was not set start on boot

k3s etc won't initialise unless you start using the APPS or is manually invoked at the CLI. Looks like you'll have to re-install SCALE.

Networking is temperamental. I started with DHCP on the NIC as per default setup, then remove IP and created a bridge with static IP and nic as a member BEFORE attempting to save he new config. When it goes into the "test" state, open a new browser tab to login on the static IP and save the new network config. Or you can setup a static IP at the console, immediately after installing SCALE.

Ok ,reinstalled and I have not selected the pool for application. In this way with the bridge set up everything is working.
Hope they solve this bug before release

 

[morganL]

Ok ,reinstalled and I have not selected the pool for application. In this way with the bridge set up everything is working.
Hope they solve this bug before release

P{lease report the bug.... instructions on how to replicate will be appreciated.