TrueCharts Integrates Docker Compose with TrueNAS SCALE

[JoshDW19]

TrueCharts Integrates Docker Compose with TrueNAS SCALE

TrueNAS SCALE is scale-out storage and hyperconverged infrastructure that uses Kubernetes for deploying containerized (e.g. Docker) applications. Kubernetes allows single containers or pods of containers to be easily deployed as Helm Charts on a unified infrastructure. Third party App catalogs such as TrueCharts provide a large, pre-tested library of applications built using Helm Charts.

Docker Compose is a very popular orchestration tool that has been used for many years by the Docker Community. However, Docker Compose and Kubernetes do not co-exist easily on a single system. Until now, pre-existing Docker Compose files (yml) were not supported on TrueNAS SCALE without breaking recommended operating guidelines. Users would have to run a separate Linux VM with Docker to support any Compose-based applications. This would consume extra CPU and RAM and would make the operator experience much less consistent.

TrueCharts has removed this limitation with a new Docker-Compose App that is ready for testing. The App is downloaded and runs like any other application. While it is being loaded, the user provides the Compose file that describes their docker-compose application, its networking and storage. Effectively, TrueCharts has provided a “Compose in Kubernetes” capability.

So how does this “Compose in Kubernetes” magic work?

TrueCharts applies the technology known as ’Docker in Docker’, which is effectively a Docker container that runs the Docker run-time environment. It provides the storage and networking within this runtime environment using the storage and networking resources provided to the Docker-Compose app.

Docker uses this technology for their own development and testing. In this TrueNAS scenario, it’s constrained to only load one compose-file automatically. The primary restriction is that Compose apps cannot have networking that conflicts with any of the Kubernetes networks, as would be expected.

The Docker and Kubernetes networking models are different and so the applications work more easily if they are separated and not networked together. It's best to use Docker-Compose App as a separated Docker environment for stand-alone applications.

Benefits to TrueNAS SCALE Community

Now TrueNAS SCALE can run any combination of Kubernetes apps described by Helm Charts, Docker containers, and Docker Compose applications. TrueNAS SCALE Apps and docker-compose are different and separated ways of using containers, yet still with all the efficiencies of shared storage and compute. Docker-Compose services persist through software updates, as well as reboots.

It's Time to Kick the Tires

Truecharts released the Docker-Compose App on March 6. Internal testing and documentation of the App has been completed. A short video tutorial is also now available.

For those of you with both Docker-Compose and TrueNAS SCALE experience, please download the Docker-Compose App and try it out with your favorite Compose files. Let us know where it succeeded and where there were issues. Please also join the TrueCharts Discord channel to chat with their developers if you find any issues.

Want to know more about TrueCharts? Visit TrueCharts.org.

 

[radomirpolach]

Testing it and Docker-Compose is unable to connect to my Virtual Servers (run within TrueNAS), but is able to connect to launched docker images or local network.

 

[sretalla]

Testing it and Docker-Compose is unable to connect to my Virtual Servers (run within TrueNAS), but is able to connect to launched docker images or local network.

Maybe related to bridge setup? (or lack of it)

SOLVED - KVM Guest Cannot Ping TrueNAS SCALE Host (or vice-versa)

I created an Ubuntu VM with the default settings. This bridged the VM directly into my network (192.168.87.217). The FreeNAS host is 192.168.87.216. Neither can ping each other. I wanted to setup NFS on the FreeNAS host and have the Ubuntu VM mount the NFS share to access the files, but no...

www.truenas.com

 

[radomirpolach]

So far no solution, but I hit another issue.

How to set Docker-Compose app to work with Traefik? Nothing I set there doesn't seems to work.

 

[radomirpolach]

Sincerely, you need to be quite an expert to set up Docker Compose App with Traefic. Finally, done.

Without kubectl terminal, I would not be able to that.

 

[morganL]

Sincerely, you need to be quite an expert to set up Docker Compose App with Traefic. Finally, done.

Without kubectl terminal, I would not be able to that.

Writing up what you did is always useful for both other users and to give us ideas on what can be done better (product or documentation).

 

[radomirpolach]

Writing up what you did is always useful for both other users and to give us ideas on what can be done better (product or documentation).

Yes, I will put together some info.

Btw, why is docker image launched through some special means and doesn't have Docker App as well?

Because I want to run my docker images with Traefik ingress and I have to use compose which adds a layer of docker in docker? Or is External Service App suppose to act as middleware between run docker image and Traefik?

 

[radomirpolach]

Setting up Docker Compose app with Traefik:

For each port, you want to expose to Traefik, you need to setup a custom service in Configure Add Manual Custom Services.

In my case, I have a Docker Comose App I call alpha (name_of_application) with exposed port 9093 which is a HTTP service I want to handle by Traefic ingress.

First configure custom service:

Custom service
Name = alpha (name_of_service)
ServiceType = ClusterIP
CustomPorts
PortName = alpha-port
PortType = TCP
TargetPort = 9093
ContainerPort = 9093
NodePort = <left empty>

Then add Ingress and you need to configure hosts:
Custom ingress
Name = alpha-ingress
Configure hosts
Host
HostName = alpha.example.com
Configure paths
Host
path = /
pathType = Prefix
ServiceName = alpha-docker-compose-alpha (This is most important part, it is name_of_application-docker-compose-name_of_service.)
ServicePort = 9093

Then add TLS settings and you need to configure hosts:
Configure TLS-Settings
Host
Configure Certificate Hosts
Host = alpha.example.com
Select TrueNAS SCALE Certificate = <select appropriate certificate for host>

 

[truecharts]

Sincerely, you need to be quite an expert to set up Docker Compose App with Traefic. Finally, done.

Without kubectl terminal, I would not be able to that.

This is only logical, because we. officially, do not actually support that at all ;-)

 

[radomirpolach]

This is only logical, because we. officially, do not actually support that at all ;-)

I read it at least eleven times already on this forum that something is not supported. :D

The messed up part here is that the name of the service is not the actual name, but there is some autogenerated name based on this name in this format: name_of_application-docker-compose-name_of_service.

It should be mentioned in the description of the field or, better yet, there should be a dropdown menu of available services.

But I am still having issues that I can't connect from TrueNAS to kvm virtual servers which blocks me from wrapping them by External Service. The issue is outside of kubernetes as I can't ping them from TrueNAS itself. There is either some routing or some firewall issue.

 

[morganL]

I read it at least eleven times already on this forum that something is not supported. :D

The messed up part here is that the name of the service is not the actual name, but there is some autogenerated name based on this name in this format: name_of_application-docker-compose-name_of_service.

It should be mentioned in the description of the field or, better yet, there should be a dropdown menu of available services.

But I am still having issues that I can't connect from TrueNAS to kvm virtual servers which blocks me from wrapping them by External Service. The issue is outside of kubernetes as I can't ping them from TrueNAS itself. There is either some routing or some firewall issue.

Are you follwoing the bridge instructions?

Accessing NAS From a VM |

Accessing NAS From a VM

www.truenas.com

 

[radomirpolach]

Haven't seen those, but I tried to configure bridge myself, will try, but already hit a snag, when trying to do the setup, I get:

The network MY_IP/24 is already in use by another interface.

So I need to remove IP address from the original interface first.

The second issue (I have already done that myself) is that when I apply new configuration with br0, remove IP from the original interface and add the IP to br0 I loose connectivity when testing the change.

 

[radomirpolach]

https://user-images.githubusercontent.com/2256039/165262951-5ed0bc4a-61ae-480b-8efe-4f71ccf3a0db.png

Testing changes fails.

 

[radomirpolach]

Btw, IP address for the screenshot purposes is dummy.

 

[truecharts]

I read it at least eleven times already on this forum that something is not supported. :D

Well, at least from our side everything under an advanced or expert tag is not supported, which means we do not offer support by our support staff for said features.

 

[jimmyhiggs337]

Ok, I followed the tutorial, wrote the example dockerfile verbatim, tested on my pc first, it ran just fine, I moved it to my server, it ran at first. I removed the compose app, moved the compose file into folder named "portainer" sudenly I'm getting the error:
level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}. Err :connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc

 

[jimmyhiggs337]

Ok, I followed the tutorial, wrote the example dockerfile verbatim, tested on my pc first, it ran just fine, I moved it to my server, it ran at first. I removed the compose app, moved the compose file into folder named "portainer" sudenly I'm getting the error:
level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}. Err :connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc

And I of course changed the "data" file path to the correct location first

 

[jimmyhiggs337]

Ok, I followed the tutorial, wrote the example dockerfile verbatim, tested on my pc first, it ran just fine, I moved it to my server, it ran at first. I removed the compose app, moved the compose file into folder named "portainer" sudenly I'm getting the error:
level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}. Err :connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc

Update, I fully remade my dockerfile, it now works, I have no idea why the old one worked on my pc but not my server, but, this new one is working, and I'm not going to look a gift horse in the mouth lol.

 

[LV52]

Setting up Docker Compose app with Traefik:

For each port, you want to expose to Traefik, you need to setup a custom service in Configure Add Manual Custom Services.

In my case, I have a Docker Comose App I call alpha (name_of_application) with exposed port 9093 which is a HTTP service I want to handle by Traefic ingress.

First configure custom service:

Custom service
Name = alpha (name_of_service)
ServiceType = ClusterIP
CustomPorts
PortName = alpha-port
PortType = TCP
TargetPort = 9093
ContainerPort = 9093
NodePort = <left empty>

Then add Ingress and you need to configure hosts:
Custom ingress
Name = alpha-ingress
Configure hosts
Host
HostName = alpha.example.com
Configure paths
Host
path = /
pathType = Prefix
ServiceName = alpha-docker-compose-alpha (This is most important part, it is name_of_application-docker-compose-name_of_service.)
ServicePort = 9093

Then add TLS settings and you need to configure hosts:
Configure TLS-Settings
Host
Configure Certificate Hosts
Host = alpha.example.com
Select TrueNAS SCALE Certificate = <select appropriate certificate for host>

Hi, where can i setup a custom service in Configure? I couldn't find the "custom service" in the configure of docker-compose. Thanks!

 

[notanumba]

Update, I fully remade my dockerfile, it now works, I have no idea why the old one worked on my pc but not my server, but, this new one is working, and I'm not going to look a gift horse in the mouth lol.

hi, i stumbled in the same error. Could you post your working yaml file ?

here is my log:

2023-01-21 22:52:05.930331+00:00time="2023-01-21T22:52:05.930012337Z" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}. Err :connection error: desc = \"transport: Error while dialing dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc

an my file:

version: '2.4'

networks:
privatenetwork:
name: privatenetwork
driver: bridge
ipam:
driver: default
config:
- subnet: 192.168.10.0/24
gateway: 192.168.10.1

services:
portainer:
read_only: true
pids_limit: 4096
mem_limit: 4g
cpus: 3
cpu_shares: 1024
image: portainer/portainer-ce:latest
healthcheck:
disable: true
container_name: portainer
restart: on-failure:5
security_opt:
- no-new-privileges:true
networks:
- privatenetwork
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /mnt/ssdapps/docker_data/compose/data:/data:rw
ports:
- "2001:9943"