docker compose?

[dragon2611]

If using the internal url make sure trafik and cloudflared have a common docker network in their compose files.

Also you may need to configure cloudflared to trust a self signed cert if you are using one

 

[Ixian]

If using the internal url make sure trafik and cloudflared have a common docker network in their compose files.

Also you may need to configure cloudflared to trust a self signed cert if you are using one

They are already skipping cert verification via noTLSVerify: true

The network part could be tricky. The Compose app itself sits in the same network as the rest of the apps and you can access them via their internal DNS. If you run Traefik (or another RP) inside Compose with Cloudflared, etc. then it should be simple; where things may get tricky is if the Traefik app (the Truecharts one) is used instead. I'm not sure how to leverage that with Compose in this setup.

 

[bitbit]

They are already skipping cert verification via noTLSVerify: true

The network part could be tricky. The Compose app itself sits in the same network as the rest of the apps and you can access them via their internal DNS. If you run Traefik (or another RP) inside Compose with Cloudflared, etc. then it should be simple; where things may get tricky is if the Traefik app (the Truecharts one) is used instead. I'm not sure how to leverage that with Compose in this setup.

Exactly. Running Traefik via compose and setting Traefik tags for TrueChart Apps is likely a huge headache, if possible. I was unable to connect to Traefik via internal address but did have success connecting Cloudflared to a TrueCharts Traefik app via host IP (shown below using TrueChart Traefik's default port). All seems to be working as expected.

Code:

tunnel: CLOUDFLARED_UUID
credentials-file: /home/nonroot/.cloudflared/CLOUDFLARED_UUID.json

# forward all traffic to Reverse Proxy w/ SSL and no TLS Verify
ingress:
  - service: https://192.168.1.2:443
    originRequest:
      noTLSVerify: true

 

[Ixian]

If the Truecharts Traefik app allows external host forwarding like vanilla Traefik does (it should, but I haven't gone through the app options to figure out how, I'm used to using yaml files for this) there may be a way to have it work with containers run via the Compose app.

 

[truecharts]

Few other things I've noted:

By default /mnt, /root, and /cluster are exposed from the host to the Compose app. Those should cover most use cases however it can get a little tricky if you need to access other host resources.

For example, one of my test containers needs read-only access to /etc/timezone. This doesn't exist in the Compose app and /etc isn't passed through by default from the host so initially that will fail. You can bind /etc/timezone in read-only mode to the Compose app though, which in turn will make it available to bind to a container, so this shouldn't be an issue in general.

Where things might get tricky is with GPU passthrough. I don't have a GPU in my test setup so I am not sure if I can pass through /dev/dri or if there are other impediments to getting GPU passthrough to work with docker-compose containers run through this App, or if that will conflict with other Apps, etc.

I'm also still working through how containers stood up via the Compose App can interact with other Apps. Since it's using the K3S network stack you can ping apps via their internal kubernetes domain name i.e. ping sabnzbd.ix-sabnzbd.svc.cluster.local works from the Compose shell to reach an app named sabnzbd. I assume this means we can link Compose container apps with SCALE apps but haven't done much testing.

And I'm not sure how it is leveraging the host docker socket, host resource allocation, etc.

Still, exciting to see this and hopefully it turns out to be a good solution for running Compose.

On the topic of timezone, that should be readable. But if it's not we're open to add that.

On the topic of GPU passthrough: just use passthrough for the SCALE Apps as normal and that should (tm) make /dev/dri available for docker-compose to passthrough yet-again.

We do not advice trying to connect Docker-Compose to the Apps system internally at all. We don't test it and there are many possible caveats.

It's not leveraging the host docker socket in any way, shape or form.
As explained in the quick-start and blogpost, it's a completely isolated docker daemon.

and setting Traefik tags for TrueChart Apps is likely a huge headache, if possible.

You CANNOT in any way, shape or form tag SCALE Apps to go through docker-compose.
Like explained in the manual, you really need to view the docker-compose App as a completely(!) seperate system.

If the Truecharts Traefik app allows external host forwarding like vanilla Traefik does (it should, but I haven't gone through the app options to figure out how, I'm used to using yaml files for this) there may be a way to have it work with containers run via the Compose app.

Our Traefik App is just a dedicated Traefik Ingress controller, so it has no such setting itself.

We do have an external-service App to pass through external services (IP:Port combinations), this might indeed also be compatible with exposed Apps from Docker-Compose, but we're not going to give guarantees or support on that.

 

[morganL]

Some more discussion on the Docker-Compose App from TrueNAS SCALE is below. It's a great example of a forum discussion (this thread) turning into really useful code. Looking forward to seeing the results.

TrueCharts Integrates Docker Compose with TrueNAS SCALE

TrueCharts Integrates Docker Compose with TrueNAS SCALE TrueNAS SCALE is scale-out storage and hyperconverged infrastructure that uses Kubernetes for deploying containerized (e.g. Docker) applications. Kubernetes allows single containers or pods of containers to be easily deployed as Helm...

www.truenas.com

 

[SnoppyFloppy]

For my Truechart apps I prefer not to expose their port to LAN and only access them trough the traefik reverse proxy.
Does anyone know if it is possible to do the same with services deployed with the docker-compose apps. I.e. they are accessible to traefik but not by ip:port form LAN. Any ideas?

 

[truecharts]

As stated in the docs, you really need to view docker-compose (App) and SCALE Apps as two totally different systems.
Definately when it comes to networking.

 

[SnoppyFloppy]

Yeah I know, I’ve read the docs but I figured that maybe there’s a docker functionality to only expose a service to localhost where truecharts external services could pick it up.

 

[bitbit]

Yeah I know, I’ve read the docs but I figured that maybe there’s a docker functionality to only expose a service to localhost where truecharts external services could pick it up.

That would require internal communication. As @truecharts has stated, they are two different systems.

 

[Stux]

Great news about the Docker-compose support in TrueNAS Scale :)

I ended up going down the Kubernetes rabbit hole for a bit. Still needed that Compose support

Looking forward to evaluating Scale now, and eventually transitioning to it, and then transitioning *some* of the Compose apps that we run/built to Kubernetes/TrueCharts etc

 

[Jip-Hop]

Wow, the TrueCharts Docker-Compose App looks like it will solve my Jira issue!

For the same reason no guarantees are given by iX Systems either way (removing or keeping) when it comes to docker-compose.
It simply just "happens to be there" due to the chosen container backend for kubernetes.

The hacks to make docker-compose work on SCALE were not guaranteed to be future proof. As far a I understand the Docker-Compose App has no such issues But just for the sake of clarity: how future proof is the TrueCharts Docker-Compose App?

 

[truecharts]

Wow, the TrueCharts Docker-Compose App looks like it will solve my Jira issue!

The hacks to make docker-compose work on SCALE were not guaranteed to be future proof. As far a I understand the Docker-Compose App has no such issues But just for the sake of clarity: how future proof is the TrueCharts Docker-Compose App?

It's fully isolated and kubernetes native by design, so it's future is at least not connected to choices made by iX-Systems. :)

 

[radomirpolach]

It's fully isolated and kubernetes native by design, so it's future is at least not connected to choices made by iX-Systems. :)

I have an issue that my Docker-Compose App can't reach Virtual Server run on TrueNAS, not even ping. But it can ping devices on my local network on launched docker images. What is the issue? I do not expect anything fancy, but I would expect that connection to IP would work without issues.

 

[truecharts]

I have an issue that my Docker-Compose App can't reach Virtual Server run on TrueNAS, not even ping. But it can ping devices on my local network on launched docker images. What is the issue? I do not expect anything fancy, but I would expect that connection to IP would work without issues.

Sadly enough we do not offer support here, please file a support ticket with our support staff on discord if you run into any issues.

Although it has to be said: We do not actively support connectivity between the docker-compose App and VM's, Host or other Apps.

As this has been shown to be problematic in the past with other solutions as well (for example: users where VM's cannot even talk to the host), it would be too troublesome to support users mixing in other solutions than our own.

 

[morganL]

I have an issue that my Docker-Compose App can't reach Virtual Server run on TrueNAS, not even ping. But it can ping devices on my local network on launched docker images. What is the issue? I do not expect anything fancy, but I would expect that connection to IP would work without issues.

These things can be made to work... via bridge interfaces. You would have to describe your exact set-up before anyone can help identify the issue.

 

[Jip-Hop]

Has anyone tried using named volumes (instead of bind mounts) from Portainer using the docker-compose app? If so, where are they stored? I assume /var/lib/docker/volumes is inside the docker-compose pod and not stored on my storage pool... If that's the case then how to access the contents of these volumes for backup purposes?

 

[truecharts]

Has anyone tried using named volumes (instead of bind mounts) from Portainer using the docker-compose app? If so, where are they stored? I assume /var/lib/docker/volumes is inside the docker-compose pod and not stored on my storage pool... If that's the case then how to access the contents of these volumes for backup purposes?

You don't have to use docker-volumes, it's generally adviced against and there is no good reason to do so anyway.
You should rather use hostpath mounts to /mnt/poolname from within the app

 

[Jip-Hop]

The docker-compose app is working really well! Thanks TrueCharts Very glad there's now a future proof way to work with docker-compose and run Portainer on SCALE. However I noticed high CPU usage and power draw caused by the k3s service. I could stop the k3s.service, which reduces CPU usage. But for now I think I'll go back to the old hack to run docker-compose on SCALE. I case this hack stops working I now know I can just start the docker-compose app with my compose file and I'll have Portainer (and my containers configured as stacks) back :)

 

[DirkTas67]

If you go the PV route you can fire up the app (so the volume gets created and claimed) then from the cli run k3s kubectl describe pv and in the VolumeAttributes you should find the path to it on the filesystem (it'll be inside the pool location you specified for Apps when you set it up). You can copy existing data there - make sure you change any permissions on your data as needed (i.e. to the guid/puid the App runs at, etc. - it's an option in the app config, with Truecharts apps).

@Ixian

Name: pvc-fad093e4-4a9d-4916-9852-13485e1b7f30
Labels: <none>
Annotations: pv.kubernetes.io/provisioned-by: zfs.csi.openebs.io
Finalizers: [kubernetes.io/pv-protection]
StorageClass: ix-storage-class-esphome
Status: Bound
Claim: ix-esphome/esphome-config
Reclaim Policy: Retain
Access Modes: RWO
VolumeMode: Filesystem
Capacity: 999Gi
Node Affinity:
Required Terms:
Term 0: openebs.io/nodeid in [ix-truenas]
Message:
Source:
Type: CSI (a Container Storage Interface (CSI) volume source)
Driver: zfs.csi.openebs.io
FSType: zfs
VolumeHandle: pvc-fad093e4-4a9d-4916-9852-13485e1b7f30
ReadOnly: false
VolumeAttributes: openebs.io/cas-type=localpv-zfs
openebs.io/poolname=apps/ix-applications/releases/esphome/volumes
storage.kubernetes.io/csiProvisionerIdentity=1654201089570-8081-zfs.csi.openebs.io

according to this information the PVC folder of the /config mount is in the /ix-applications/releases/esphome/volumes folder, when I look there there is nothing but another folder. Any ideas?