Docker Containers Have No Outbound Internet Connectivity?

[drinny]

I just started playing around with TrueNAS Scale today. I've been itching to move to containerized applications instead of managing them individually in jails within TrueNAS Core.

So far, I've managed to get a docker-compose.yml file built with all of my containers configured within. The containers run just fine aside from the fact that they don't seem to have outbound Internet access? Some of the apps need to be able to contact outside hosts and none seem to be able to by default?

The only way that I've found around this is to:

• edit /etc/docker/daemon.json
• change "iptables": false to "iptables":true
• sudo systemctl restart docker

This does allow my containers to then reach the internet, but this needs to be manually done on server reboot.

Is there a better way to go about doing this? I'm guessing that something may be a little more polished down the road once Apps are more fully supported in Scale. I'm just looking on how I might bridge the gap in the mean time.

 

[ornias]

That solution you just posted, IS the stop-gap.

Just to be clear: Docker-compose is NOT supported, it's allowed and not prevented to use, but it's not actually supported.

I think you're also misunderstanding that "Apps" refer only to TrueNAS SCALE Apps, the Apps section in the GUI and theoption in there to add docker containers. It does NOT refer to docker-compose.

Simply put:
docker-compose is not removed or prevented, but you're on your own if you want to use non-IX and/or non-k8s solutions.

 

[drinny]

Understood that docker-compose is not supported. I also understand that "Apps" refers to the official TrueNAS SCALE Apps in the GUI there and is not related to docker-compose on the CLI.

I'm glad that you mentioned that there is an option in there to add docker containers. I only saw that there was a Catalog there and assumed that I was limited to those containers for the time being. I didn't know that there already exist a way from within the GUI to add containers manually.

I just spun up a container using the GUI and it had no issues connecting to the outside world which is excellent! My only gripe (other than the GUI being a little slower to setup than docker-compose via CLI) would be that I'm limited to binding to ports >9000, but I think I've read that is being looked at to see if it can be changed.

 

[ornias]

My only gripe (other than the GUI being a little slower to setup than docker-compose via CLI) would be that I'm limited to binding to ports >9000, but I think I've read that is being looked at to see if it can be changed.

I don't know where you read that, but thats not true afaik. It's ALREADY a severely lower port minimum than default k3s allows.
remember: Nodeports are not really how k8s/k3s is supposed to be used (rather with ingresses and loadbalancers)

 

[drinny]

Oops. I should have written <9000, not >9000. Trying to specify a nodeport of say 8999 will bring up an error in the GUI that "Minimum value is 9000."

It's not necessarily a deal breaker for me. I'll just have to map to higher number ports and update my bookmarks. I won't pretend to know why k8s/k3s recommend such a high range. Ultimately, my Scale server is just going to be a standalone server for home use, so more complicated network setups with ingresses and loadbalancers likely will never make it into my network here. Scale is really likely overkill for my use case. If Core was based on linux and could have run containers natively on the host there, I'd probably still be on Core.

Thanks for all your replies.

 

[ornias]

@drinny people should stop being scared of things like ingresses, because if you don't know something you can hardly evaluate if something is hard or not ;-)

Its nothing more than an automated way of adding containers to a reverse proxy in most usecases.

 

[no12358]

I just started playing around with TrueNAS Scale today. I've been itching to move to containerized applications instead of managing them individually in jails within TrueNAS Core.

So far, I've managed to get a docker-compose.yml file built with all of my containers configured within. The containers run just fine aside from the fact that they don't seem to have outbound Internet access? Some of the apps need to be able to contact outside hosts and none seem to be able to by default?

The only way that I've found around this is to:

• edit /etc/docker/daemon.json
• change "iptables": false to "iptables":true
• sudo systemctl restart docker

This does allow my containers to then reach the internet, but this needs to be manually done on server reboot.

Is there a better way to go about doing this? I'm guessing that something may be a little more polished down the road once Apps are more fully supported in Scale. I'm just looking on how I might bridge the gap in the mean time.

do bridge networks work when you set it to true? host networks work with enabling this.

have you found a way to enable this on server boot?

 

[Ixian]

Rolling your own Compose setup is tolerated, not supported, in SCALE. Ix Systems has reserved the right to remove it (docker-compose and/or the docker package itself) in future distro updates - this seems unlikely but not unthinkable. Official app support is provided with the K8s backed app solution packaged in SCALE's middleware and GUI. It supports external app lists from maintaners like Truecharts.

Since SCALE is based off Debian and the relevant stable packages are included (Docker 20.10.11 && Docker-compose 1.25) it is possible to roll your own setup if you like. Don't expect to get support other than from other SCALE users (for SCALE-specific issues) though of course you'll have a much larger community for support with Compose, etc.

You can script changes to make life easier. This one makes a good base to build off of, though I set it up as a post-init, not pre-init task: https://gist.github.com/Jip-Hop/af3b7a770dd483b07ac093c3b205323f

The above compares the running docker daemon.json file to a custom one you create (to enable ip tables support, change the default data directory if needed, etc.) and over-writes it/restarts docker if your custom file isn't in use. You can obviously do a bunch of things this way - check the version of Compose and upgrade it if it is below a certain version, for example (I run Compose 1.29.2 for various reasons). All of this, of course, is unsupported, but if you are a home user and don't have a paid support subscription you are already unsupported outside of the community anyway.

I use SCALE for ZFS storage and storage management. It also has much better VM support than Core in the form of KVM, with "official" GUI & Middleware support. I appreciate the compact, memory-resident curated distro that I don't have to spend nearly as much time maintaining vs. "DYI" NAS setups based off Ubuntu, or VM managers like Proxmox, etc. And the GUI-based management tools for things like replication, permissions, and so on are top notch.

I just don't like the direction they went on the "app" side of things, so I don't use it. I already know how to work with Compose and have a nice integrated setup with Traefik, etc. Could I learn how to map that over to how SCALE manages "apps"? Of course. I just don't see the point to it. There's nothing it does, or is planning to do, that I couldn't do more easily, based on the applications I run. Others will arrive at a different conclusion based on their needs and experience and that, of course, is just fine. This isn't a zero-sum game.

 

[no12358]

Rolling your own Compose setup is tolerated, not supported, in SCALE. Ix Systems has reserved the right to remove it (docker-compose and/or the docker package itself) in future distro updates - this seems unlikely but not unthinkable. Official app support is provided with the K8s backed app solution packaged in SCALE's middleware and GUI. It supports external app lists from maintaners like Truecharts.

Since SCALE is based off Debian and the relevant stable packages are included (Docker 20.10.11 && Docker-compose 1.25) it is possible to roll your own setup if you like. Don't expect to get support other than from other SCALE users (for SCALE-specific issues) though of course you'll have a much larger community for support with Compose, etc.

You can script changes to make life easier. This one makes a good base to build off of, though I set it up as a post-init, not pre-init task: https://gist.github.com/Jip-Hop/af3b7a770dd483b07ac093c3b205323f

The above compares the running docker daemon.json file to a custom one you create (to enable ip tables support, change the default data directory if needed, etc.) and over-writes it/restarts docker if your custom file isn't in use. You can obviously do a bunch of things this way - check the version of Compose and upgrade it if it is below a certain version, for example (I run Compose 1.29.2 for various reasons). All of this, of course, is unsupported, but if you are a home user and don't have a paid support subscription you are already unsupported outside of the community anyway.

I use SCALE for ZFS storage and storage management. It also has much better VM support than Core in the form of KVM, with "official" GUI & Middleware support. I appreciate the compact, memory-resident curated distro that I don't have to spend nearly as much time maintaining vs. "DYI" NAS setups based off Ubuntu, or VM managers like Proxmox, etc. And the GUI-based management tools for things like replication, permissions, and so on are top notch.

I just don't like the direction they went on the "app" side of things, so I don't use it. I already know how to work with Compose and have a nice integrated setup with Traefik, etc. Could I learn how to map that over to how SCALE manages "apps"? Of course. I just don't see the point to it. There's nothing it does, or is planning to do, that I couldn't do more easily, based on the applications I run. Others will arrive at a different conclusion based on their needs and experience and that, of course, is just fine. This isn't a zero-sum game.

Removing docker seems like a monumentally bad idea... I want to install a reverse proxy so I can host my sites jelly etc on the same server and easily get SSL certificates.

Why even offer apps if you then restrict stuff.

Edit:
Just tested, if you unset pool in the setting of apps section, the daemon.json doesn't get overwritten on reboot. Bridge networks work, so I'm all set. Hopefully they leave it alone in the future update.

 

[Ixian]

Removing docker seems like a monumentally bad idea... I want to install a reverse proxy so I can host my sites jelly etc on the same server and easily get SSL certificates.

Why even offer apps if you then restrict stuff.

Edit:
Just tested, if you unset pool in the setting of apps section, the daemon.json doesn't get overwritten on reboot. Bridge networks work, so I'm all set. Hopefully they leave it alone in the future update.

You're confusing where self-rolled docker setups via CLI fit in to Ix's app strategy - short answer is, they don't. It's Kubernetes managed by their own custom GUI and middleware. You can install individual docker apps through the same UX. We could debate the pros, cons, & whys of this approach all day but it's a waste of time.

If you modify the daemon.json you should unset the app pool regardless as changes to how docker runs in SCALE can potentially break the official App setup. Treat it as either/or. The script is really there for convenience during system updates & such. You'll want to be careful with updates and take a config backup (which you should be doing anyway) as there's no guarantee your custom setup won't break due some unforeseen issue. That's also what they mean by unsupported.

Right now it's really not that big a deal.

 

[no12358]

You're confusing where self-rolled docker setups via CLI fit in to Ix's app strategy - short answer is, they don't. It's Kubernetes managed by their own custom GUI and middleware. You can install individual docker apps through the same UX. We could debate the pros, cons, & whys of this approach all day but it's a waste of time.

If you modify the daemon.json you should unset the app pool regardless as changes to how docker runs in SCALE can potentially break the official App setup. Treat it as either/or. The script is really there for convenience during system updates & such. You'll want to be careful with updates and take a config backup (which you should be doing anyway) as there's no guarantee your custom setup won't break due some unforeseen issue. That's also what they mean by unsupported.

Right now it's really not that big a deal.

The docker install UX that truenas provides is baffling on so many levels, I can't make heads or tails of what to set to run a docker app.

 

[Ixian]

The docker install UX that truenas provides is baffling on so many levels, I can't make heads or tails of what to set to run a docker app.

You'll get no argument from me. It's built to service an as-yet unrealized, aspirational vision that incorporates app & storage scaling tucked behind a nice management interface that their Small/Medium business market - the one that pays them - could use.

For "home NAS" type users - though what constitutes a "home" type setup is often in the eye of the beholder these days - who want a nice NAS they can run a bunch of media-centric apps on to manage and stream their movies collections and maybe keep important stuff around the house backed up it's over-complicated and unwanted, particularly if you are already well versed in how docker, etc. work.

 

[hiro5id]

@drinny people should stop being scared of things like ingresses, because if you don't know something you can hardly evaluate if something is hard or not ;-)

Its nothing more than an automated way of adding containers to a reverse proxy in most usecases.

I have no problem learning to use kubernetes yaml files and ingresses.
My issue is that I have a kubernetes pod and I would like to mount a local ZFS dataset into it. Example (/mnt/TANK/photos) I can't seem to be able to figure out how.

 

[hiro5id]

Rolling your own Compose setup is tolerated, not supported, in SCALE. Ix Systems has reserved the right to remove it (docker-compose and/or the docker package itself) in future distro updates - this seems unlikely but not unthinkable. Official app support is provided with the K8s backed app solution packaged in SCALE's middleware and GUI. It supports external app lists from maintaners like Truecharts.

Since SCALE is based off Debian and the relevant stable packages are included (Docker 20.10.11 && Docker-compose 1.25) it is possible to roll your own setup if you like. Don't expect to get support other than from other SCALE users (for SCALE-specific issues) though of course you'll have a much larger community for support with Compose, etc.

You can script changes to make life easier. This one makes a good base to build off of, though I set it up as a post-init, not pre-init task: https://gist.github.com/Jip-Hop/af3b7a770dd483b07ac093c3b205323f

The above compares the running docker daemon.json file to a custom one you create (to enable ip tables support, change the default data directory if needed, etc.) and over-writes it/restarts docker if your custom file isn't in use. You can obviously do a bunch of things this way - check the version of Compose and upgrade it if it is below a certain version, for example (I run Compose 1.29.2 for various reasons). All of this, of course, is unsupported, but if you are a home user and don't have a paid support subscription you are already unsupported outside of the community anyway.

I use SCALE for ZFS storage and storage management. It also has much better VM support than Core in the form of KVM, with "official" GUI & Middleware support. I appreciate the compact, memory-resident curated distro that I don't have to spend nearly as much time maintaining vs. "DYI" NAS setups based off Ubuntu, or VM managers like Proxmox, etc. And the GUI-based management tools for things like replication, permissions, and so on are top notch.

I just don't like the direction they went on the "app" side of things, so I don't use it. I already know how to work with Compose and have a nice integrated setup with Traefik, etc. Could I learn how to map that over to how SCALE manages "apps"? Of course. I just don't see the point to it. There's nothing it does, or is planning to do, that I couldn't do more easily, based on the applications I run. Others will arrive at a different conclusion based on their needs and experience and that, of course, is just fine. This isn't a zero-sum game.

I totally agree with this in terms of the "app" side of things.

I just want TrueNAS on Ubuntu with ZFS, and let me run and manage my own flavor of Docker stuff. I want to be able to entirely turn off the "app" stuff with the bundled k3s. Perhaps some people like that, but for others that don't, let us turn it off or have a way to remove it so that it doesn't interfere with our own Docker setup we choose.

But there does not seem to be a way to uninstall that part. I tried uninstalling Docker (so that I install my own with my own settings) but I ended up breaking TrueNAS and I had to reinstall from scratch. Luckily TrueNAS makes it very painless to recover my precious ZFS volumes.

VM's are nice, but I really wanted to run Docker natively on TrueNAS. It doesn't have to interfere with what TrueNAS already does well, GUI for ZFS and shares management. I just want to do my own thing when it comes to Docker.

 

[Kieros]

That solution you just posted, IS the stop-gap.

Just to be clear: Docker-compose is NOT supported, it's allowed and not prevented to use, but it's not actually supported.

I think you're also misunderstanding that "Apps" refer only to TrueNAS SCALE Apps, the Apps section in the GUI and theoption in there to add docker containers. It does NOT refer to docker-compose.

Simply put:
docker-compose is not removed or prevented, but you're on your own if you want to use non-IX and/or non-k8s solutions.

Hi I am using the launch docker image at the app section. I have some apps working but some apps need host network access.
But I find there is not much documentation and unaswered questions through forum.
I need to put in net=host because some apps need to be on the home network. How do I do that? how do I add stuff that is normally in the command lines when a container is started? I am searching for days on the internet how to do this... Or what route do I need to have access to the network? Perhaps I am using wrong search terms but I can not find any good explanation on how to use it.

The only thing I found was a solution here on forums by someone, but I have the feeling it is wrong.
They put in a static ip for example 192.168.8.11 for the container.
Then a static route was created by 0.0.0.0/1 and 128.0.0.0/1 and then it works. The fun part is 192.168.8.11 can be pinged from windows but it is not reachable on the designated port. Then the host ip 192.168.8.2 (truenas host) with the designated port 9004 is reachable and connected with the network. However the IP 192.168.8.11 is also taken up on the network?

Care to explain how the gui should be used to add docker variables that need to be put in? like for example --net=host?

 

[sretalla]

Care to explain how the gui should be used to add docker variables that need to be put in? like for example --net=host?

Tick this box.

how do I add stuff that is normally in the command lines when a container is started?

Add items here:

 

[Kieros]

Tick this box.

View attachment 69116


Add items here:
View attachment 69117

The box is not there to tick. instead I have this?

I do no where to put the evironment settings, but where can I put other stuff? and how to do it?
Can you give me a example for example of qbittorrent or something else like emby or jellyy fin?

 

[Kieros]

I mean I do know where to put environment stuff but all the other stuff I sometimes see.

 

[Kieros]

Should this be it? for host?

 

[sretalla]

Should this be it? for host?

No. That's not anything like what I said.

You don't define an IP address to use host networking, just do nothing in that section.