Kubernetes advanced options

[NugentS]

Under Kubernetes Settings / Advanced Settings there is "Node IP", "Route v4 Interface" and "Route v4 Gateway". If I alter Route V4 gateway to point to a different router on my LAN (it goes out through a VPN Firewall) and type curl if(p)config.io into say a Sonarr container I get an external address on the VPN connection. The main scale gateway is pointing at my primary router. But I get an address on the VPN router connection - which is good. BUT traffic out on the internet coming into the VPN does not go anywhere. It just vanishes - because its going to the primary gateway rather than back to the VPN gateway. As soon as I change the scale gateway then the port shows as open from outside down the VPN connection.

Any thoughts?

The problem is that changing the TrueNAS Scale gateway changes the default IP for any containers. I can't even use a different VLAN as Scale can only have a single default IP. I could run a VM with the containers BUT then I get all the access permissions grief trying to get access to the Scale Storage

With Core, each Jail has its own network stack, so I could do what I wanted - not so with Scale

 

[truecharts]

You can already directly mount interfaces to the Official Launch-Docker option, our Custom-App option or any of our other Apps with the same option hidden away under an advanced or expert checkbox.

 

[morganL]

Under Kubernetes Settings / Advanced Settings there is "Node IP", "Route v4 Interface" and "Route v4 Gateway". If I alter Route V4 gateway to point to a different router on my LAN (it goes out through a VPN Firewall) and type curl if(p)config.io into say a Sonarr container I get an external address on the VPN connection. The main scale gateway is pointing at my primary router. But I get an address on the VPN router connection - which is good. BUT traffic out on the internet coming into the VPN does not go anywhere. It just vanishes - because its going to the primary gateway rather than back to the VPN gateway. As soon as I change the scale gateway then the port shows as open from outside down the VPN connection.

Any thoughts?

The problem is that changing the TrueNAS Scale gateway changes the default IP for any containers. I can't even use a different VLAN as Scale can only have a single default IP. I could run a VM with the containers BUT then I get all the access permissions grief trying to get access to the Scale Storage

With Core, each Jail has its own network stack, so I could do what I wanted - not so with Scale

Isn't this just the limitation of using "Host Networking"?

Yes, Jails are closer to a "lightweight VM" by default. Kubernetes provides options.

 

[NugentS]

You can already directly mount interfaces to the Official Launch-Docker option, our Custom-App option or any of our other Apps with the same option hidden away under an advanced or expert checkbox.

I'll have a look at that over the next day and see if I can make it work

 

[NugentS]

I gave that a try. I created a new container using Sonarr (as I know I can type curl ipconfig.io) on it.
All default options except the expert network. Assign an unused working NIC and

Error: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 423, in run
await self.future
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 459, in __run_body
rv = await self.method(*([self] + args))
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1129, in nf
res = await f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1261, in nf
return await func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/chart_release.py", line 467, in do_create
await self.middleware.call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1318, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1286, in _call
return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1186, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/helm.py", line 44, in helm_action
raise CallError(f'Failed to {tn_action} chart release: {stderr.decode()}')
middlewared.service_exception.CallError: [EFAULT] Failed to install chart release: Error: INSTALLATION FAILED: YAML parse error on sonarr/templates/common.yaml: error converting YAML to JSON: yaml: line 43: did not find expected key

So the questionis - is this a bug.
If Yes then whose bug. IX or Truecharts?

 

[morganL]

I gave that a try. I created a new container using Sonarr (as I know I can type curl ipconfig.io) on it.
All default options except the expert network. Assign an unused working NIC and

Error: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 423, in run
await self.future
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 459, in __run_body
rv = await self.method(*([self] + args))
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1129, in nf
res = await f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1261, in nf
return await func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/chart_release.py", line 467, in do_create
await self.middleware.call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1318, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1286, in _call
return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1186, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/helm.py", line 44, in helm_action
raise CallError(f'Failed to {tn_action} chart release: {stderr.decode()}')
middlewared.service_exception.CallError: [EFAULT] Failed to install chart release: Error: INSTALLATION FAILED: YAML parse error on sonarr/templates/common.yaml: error converting YAML to JSON: yaml: line 43: did not find expected key

So the questionis - is this a bug.
If Yes then whose bug. IX or Truecharts?

Did you use an TrueCharts App? If so, I'd start with TrueCharts.
If the official Apps are not working, start with TrueNAS.
If the TrueCharts team identify a TrueNAS issue... they aren't shy about letting us know and they are very good at telling us what is broken.

 

[NugentS]

Right the sonarr issue above is a red herring - it would appear to be a bug with the specific container rather than the system behind it. So please ignore
-----------------------
Current situation.

I have a LAN with two gateways on it.
GW1 is a VPN Only gateway access to the internet
GW2 is a unencrypted gateway access to the internet

DHCP on LAN points to GW2
Scale at the moment (as a workaround) default gateway points at GW1
[I control which computer uses the encrypted gateway using default gateway]

What I wanted to do is give the containers access to another network card and use DHCP to control the default gateway they use, or set the gateway up for each container. However it appears that this is impossible as it stands currently. Whether the feature becomes available later on is "who knows"

Well I have a workaround thats fine for testing purposes and I have another idea that might work. But for the moment I will give up on the idea.

 

[PackElend]

currently, Kubernetes can only operate within a single subnet but can you at least bind that to an VLAN interface?