How to get Docker back?

[simdim]

Hi All,
I had accidentally deleted Docker under "Virtualization" on TrueNas Scale, is there a way to manually get it back back?
The reason I had deleted it - I have a separate zpool for virtualization that was giving me kernel panics during zpool imports, so I had removed the disks in that pool and removed docker under virtualization ....

 

[simdim]

To clarify where is official Docker image that I can reinstall?
I think that is the reason I am getting [EFAULT] Kubernetes service is not running. error when attempting to install applications

 

[simdim]

Ok. Baby steps. Downgraded/upgraded got Docker VM back. Since I had to destroy the pool that docker was mapped to I no longer have /mnt/pool/Docker_Docker directory to mount RAW disk. Created directory as root and getting "/mnt/pool/Docker_Docker': Permission denied" on Docker start

What ownership/permissions do I need to have on it?

 

[DaSnipe]

You're asking Docker inside a VM questions on a TrueNAS SCALE forum, I'd help if I had an idea of what you're doing. A docker VM could be anything

 

[danb35]

got Docker VM back

What, exactly, is a "Docker VM"?

 

[simdim]

You're asking Docker inside a VM questions on a TrueNAS SCALE forum, I'd help if I had an idea of what you're doing. A docker VM could be anything

I am talking about default Docker VM
[EFAULT] internal error: process exited while connecting to monitor: 2022-08-24T16:04:21.536303Z qemu-system-x86_64: -blockdev {"driver":"file","filename":"/mnt/pool/Docker_Docker","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: Could not open '/mnt/pool/Docker_Docker': Permission denied

this is how I got here:

System was initially upgraded from Core to Scale
It is currently running TrueNAS-SCALE-22.02.3

1. I had a problem with corruption of pool named "pool" and had to remove the pool wipe disks and reconfigure.
2. By mistake I had deleted VM called Docker (default system installed)
3. Downgraded upgraded and regained VM Docker however it was not starting due to error [EFAULT] VM will not start as RAW Device: /mnt/pool/Docker_Docker device(s) are not available.
4. Created /mnt/pool/Docker_Docker as root/root and having permission error on start.

By the way k3s and docker services are running

root@freenas[/mnt/pool]# systemctl status k3s.service
● k3s.service - Lightweight Kubernetes
Loaded: loaded (/lib/systemd/system/k3s.service; disabled; vendor preset: disabled)
Active: activating (start) since Wed 2022-08-24 12:22:08 EDT; 6s ago
Docs: https://k3s.io
Process: 94376 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 94377 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 94378 (k3s-server)
Tasks: 21
Memory: 326.1M
CGroup: /system.slice/k3s.service
└─94378 /usr/local/bin/k3s server

Aug 24 12:22:14 freenas k3s[94378]: I0824 12:22:14.387646 94378 shared_informer.go:247] Caches are synced for node_authorizer
Aug 24 12:22:14 freenas k3s[94378]: I0824 12:22:14.396524 94378 apf_controller.go:322] Running API Priority and Fairness config worker
Aug 24 12:22:14 freenas k3s[94378]: I0824 12:22:14.396561 94378 cache.go:39] Caches are synced for APIServiceRegistrationController controller
Aug 24 12:22:14 freenas k3s[94378]: I0824 12:22:14.396562 94378 cache.go:39] Caches are synced for AvailableConditionController controller
Aug 24 12:22:14 freenas k3s[94378]: I0824 12:22:14.396723 94378 shared_informer.go:247] Caches are synced for cluster_authentication_trust_controller
Aug 24 12:22:14 freenas k3s[94378]: I0824 12:22:14.407927 94378 cache.go:39] Caches are synced for autoregister controller
Aug 24 12:22:14 freenas k3s[94378]: I0824 12:22:14.408023 94378 shared_informer.go:247] Caches are synced for crd-autoregister
Aug 24 12:22:15 freenas k3s[94378]: I0824 12:22:15.295879 94378 controller.go:132] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
Aug 24 12:22:15 freenas k3s[94378]: I0824 12:22:15.304313 94378 storage_scheduling.go:109] all system priority classes are created successfully or already exist.
Aug 24 12:22:15 freenas k3s[94378]: I0824 12:22:15.564440 94378 controller.go:132] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).


root@freenas[/mnt/pool]# systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/docker.service.d
└─override.conf
Active: active (running) since Wed 2022-08-24 12:24:28 EDT; 36s ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 98636 (dockerd)
Tasks: 28 (limit: 18675)
Memory: 124.0M
CGroup: /system.slice/docker.service
├─98636 /usr/bin/dockerd -H fd://
└─98689 containerd --config /var/run/docker/containerd/containerd.toml --log-level info

Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.380217664-04:00" level=warning msg="Your kernel does not support CPU realtime scheduler"
Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.380244985-04:00" level=warning msg="Your kernel does not support cgroup blkio weight"
Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.380252167-04:00" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.380670783-04:00" level=info msg="Loading containers: start."
Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.557975502-04:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP >
Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.659294277-04:00" level=info msg="Loading containers: done."
Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.704012526-04:00" level=info msg="Docker daemon" commit=224916af6c-unsupported graphdriver(s)=zfs version=dev
Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.704381067-04:00" level=info msg="Daemon has completed initialization"
Aug 24 12:24:28 freenas systemd[1]: Started Docker Application Container Engine.
Aug 24 12:24:28 freenas dockerd[98636]: time="2022-08-24T12:24:28.736851411-04:00" level=info msg="API listen on /run/docker.sock"
lines 1-24/24 (END)

 

[danb35]

By mistake I had deleted VM called Docker (default system installed)

In what "default system" is this installed? CORE doesn't give you a VM called Docker (or anything else) out of the box, and neither does SCALE--and in the case of SCALE it'd be pretty strange to do so, since it can handle Docker containers natively.

 

[simdim]

In what "default system" is this installed? CORE doesn't give you a VM called Docker (or anything else) out of the box, and neither does SCALE--and in the case of SCALE it'd be pretty strange to do so, since it can handle Docker containers natively.

Maybe I put it in there during CORE time and forgot about it, will ignore ....

On closer examination k3s cluster is not coming up though it is giving me :

Failed to configure kubernetes cluster for Applications: Missing "pool/ix-applications/docker, pool/ix-applications/releases, pool/ix-applications/k3s" dataset(s) required for starting kubernetes.​

All I see there:

root@freenas[/mnt/pool/ix-applications]# ls -l
total 1
drwxr-xr-x 5 root root 5 Aug 24 11:31 k3s

 

[danb35]

Have you set your pool under Apps -> Settings?

 

[simdim]

Have you set your pool under Apps -> Settings?

It gives me option to choose/unset pool but clicking on it does not bring up selection - is there a way to do it from CLI?

 

[danb35]

I don't know what "clicking on it does not bring up selection", but what happens if you click on Unset Pool, and then go to Choose Pool? It may be that the system has the old pool stuck somewhere.

 

[simdim]

I don't know what "clicking on it does not bring up selection", but what happens if you click on Unset Pool, and then go to Choose Pool? It may be that the system has the old pool stuck somewhere.

"Unset" option is not present every time I click on "Settings" and even when it is present it does not open a dialog. Clicking on Advanced does not open dialog ether.

 

[danb35]

So, to repeat my question:

what happens if you click on Unset Pool, and then go to Choose Pool?

 

[simdim]

So, to repeat my question:

currently I do not have "Unset pool" menu entry under Apps>Settings

As I had mentioned - clicking on any of them does not bring selection dialog.

 

[danb35]

currently I do not have "Unset pool" menu entry under Apps>Settings

...then go to a place where you do have it--Available Applications would be such a place. Then select it. Then see if "Choose Pool" is no longer grayed-out. If it isn't, try clicking it then.

 

[simdim]

...then go to a place where you do have it--Available Applications would be such a place. Then select it. Then see if "Choose Pool" is no longer grayed-out. If it isn't, try clicking it then.

That had done the trick - under "Available Applications" I was able to Unset the pool and set the new one.
k3s service came up and I was able to deploy app from catalogue.

Thank you very much for the help!

 

[danb35]

I filed a bug on this; seems Unset Pool should be available anywhere:

Log in with Atlassian account

Log in to Jira, Confluence, and all other Atlassian Cloud products here. Not an Atlassian user? Sign up for free.

jira-redirect.ixsystems.com

 

[Patrick M. Hausen]

@simdim Did you ever run FreeNAS 10 aka The Release That Must Not Be Named?

Because that did have a "default Docker VM" to easily deploy containers via the UI.

 

[simdim]

@simdim Did you ever run FreeNAS 10 aka The Release That Must Not Be Named?

Because that did have a "default Docker VM" to easily deploy containers via the UI.

I sure did on that box ... so since I am on SCALE now there is no harm of removing that VM?

 

[Patrick M. Hausen]

It's not necessary. SCALE was created running on Debian specifically to be able to run Linux containers natively.