Nov 28 05:18:23 truenas k3s[5796]: time="2022-11-28T05:18:23-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:18:25 truenas k3s[5796]: I1128 05:18:25.114918    5796 event.go:294] "Event occurred" object="ix-myinfinity/myinfinity-ix-chart" fieldPath="" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set myinfinity-ix-chart-5f5578fdbb to 1 from 0"
Nov 28 05:18:25 truenas k3s[5796]: I1128 05:18:25.182969    5796 event.go:294] "Event occurred" object="ix-myinfinity/myinfinity-ix-chart-5f5578fdbb" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: myinfinity-ix-chart-5f5578fdbb-vwscc"
Nov 28 05:18:25 truenas k3s[5796]: I1128 05:18:25.222239    5796 topology_manager.go:205] "Topology Admit Handler"
Nov 28 05:18:25 truenas k3s[5796]: E1128 05:18:25.222532    5796 cpu_manager.go:394] "RemoveStaleState: removing container" podUID="a9937f1a-8508-4c35-8b0e-b1623996f552" containerName="ix-chart"
Nov 28 05:18:25 truenas k3s[5796]: E1128 05:18:25.222633    5796 cpu_manager.go:394] "RemoveStaleState: removing container" podUID="78f7c6ab-3ab5-4257-864e-136b87111f5a" containerName="ix-chart"
Nov 28 05:18:25 truenas k3s[5796]: I1128 05:18:25.222735    5796 memory_manager.go:345] "RemoveStaleState removing state" podUID="78f7c6ab-3ab5-4257-864e-136b87111f5a" containerName="ix-chart"
Nov 28 05:18:25 truenas k3s[5796]: I1128 05:18:25.222818    5796 memory_manager.go:345] "RemoveStaleState removing state" podUID="a9937f1a-8508-4c35-8b0e-b1623996f552" containerName="ix-chart"

==> app_mounts.log <==
Nov 28 05:18:25 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-1da93e7f296f5a884b0b748809d82c257924a6d30c38abdd3e0e3774b6b15f70\x2dinit-merged.mount: Succeeded.

==> k3s_daemon.log <==
Nov 28 05:18:25 truenas k3s[5796]: I1128 05:18:25.424597    5796 reconciler.go:357] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrjzj\" (UniqueName: \"kubernetes.io/projected/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb-kube-api-access-rrjzj\") pod \"myinfinity-ix-chart-5f5578fdbb-vwscc\" (UID: \"4556faa0-5aaf-417e-9ccb-f801f1f3c5cb\") " pod="ix-myinfinity/myinfinity-ix-chart-5f5578fdbb-vwscc"
Nov 28 05:18:25 truenas k3s[5796]: I1128 05:18:25.424663    5796 reconciler.go:357] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ix-host-path-myinfinity-0\" (UniqueName: \"kubernetes.io/host-path/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb-ix-host-path-myinfinity-0\") pod \"myinfinity-ix-chart-5f5578fdbb-vwscc\" (UID: \"4556faa0-5aaf-417e-9ccb-f801f1f3c5cb\") " pod="ix-myinfinity/myinfinity-ix-chart-5f5578fdbb-vwscc"
Nov 28 05:18:25 truenas k3s[5796]: I1128 05:18:25.968462    5796 pod_container_deletor.go:79] "Container not found in pod's containers" containerID="8c36961e13e44f42949c775dd4d55cb2a7cbf54148c888ff2b151c24c2f97539"

==> app_mounts.log <==
Nov 28 05:18:25 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-1da93e7f296f5a884b0b748809d82c257924a6d30c38abdd3e0e3774b6b15f70-merged.mount: Succeeded.
Nov 28 05:18:26 truenas systemd[1]: run-docker-runtime\x2drunc-moby-8c36961e13e44f42949c775dd4d55cb2a7cbf54148c888ff2b151c24c2f97539-runc.N6nEr2.mount: Succeeded.

==> containerd.log <==
Nov 28 05:18:26 truenas dockerd[5679]: time="2022-11-28T05:18:26.314684654-08:00" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Nov 28 05:18:26 truenas dockerd[5679]: time="2022-11-28T05:18:26.314749576-08:00" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Nov 28 05:18:26 truenas dockerd[5679]: time="2022-11-28T05:18:26.314765999-08:00" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Nov 28 05:18:26 truenas dockerd[5679]: time="2022-11-28T05:18:26.314873754-08:00" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/8c36961e13e44f42949c775dd4d55cb2a7cbf54148c888ff2b151c24c2f97539 pid=17927 runtime=io.containerd.runc.v2

==> kern.log <==
Nov 28 05:18:27 truenas kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethd5bc0ac5: link becomes ready
Nov 28 05:18:27 truenas kernel: kube-bridge: port 1(vethd5bc0ac5) entered blocking state
Nov 28 05:18:27 truenas kernel: kube-bridge: port 1(vethd5bc0ac5) entered disabled state
Nov 28 05:18:27 truenas kernel: device vethd5bc0ac5 entered promiscuous mode
Nov 28 05:18:27 truenas kernel: kube-bridge: port 1(vethd5bc0ac5) entered blocking state
Nov 28 05:18:27 truenas kernel: kube-bridge: port 1(vethd5bc0ac5) entered forwarding state

==> daemon.log <==
Nov 28 05:18:27 truenas systemd-udevd[18012]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Nov 28 05:18:27 truenas systemd-udevd[18012]: Using default interface naming scheme 'v247'.

==> k3s_daemon.log <==
Nov 28 05:18:27 truenas k3s[5796]: time="2022-11-28T05:18:27-08:00" level=info msg="Will attempt to re-write config file /mnt/tank/ix-applications/docker/containers/8c36961e13e44f42949c775dd4d55cb2a7cbf54148c888ff2b151c24c2f97539/resolv.conf as [nameserver 192.168.1.1]"

==> app_mounts.log <==
Nov 28 05:18:28 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-b013d5be29aaa1798ce15636322a6b7229d90d3a3e3afa474c3970f05b7fd3b9\x2dinit-merged.mount: Succeeded.

==> k3s_daemon.log <==
Nov 28 05:18:27 truenas k3s[5796]: I1128 05:18:27.825029   18001 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ix-myinfinity", Name:"myinfinity-ix-chart-5f5578fdbb-vwscc", UID:"4556faa0-5aaf-417e-9ccb-f801f1f3c5cb", APIVersion:"v1", ResourceVersion:"98421", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [172.16.0.97/16] from ix-net

==> daemon.log <==
Nov 28 05:18:27 truenas systemd-udevd[18012]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Nov 28 05:18:28 truenas ntpd[2676]: bind(30) AF_INET6 fe80::7c18:b5ff:fe18:e024%13#123 flags 0x11 failed: Cannot assign requested address
Nov 28 05:18:28 truenas ntpd[2676]: unable to create socket on vethd5bc0ac5 (21) for fe80::7c18:b5ff:fe18:e024%13#123
Nov 28 05:18:28 truenas ntpd[2676]: failed to init interface for address fe80::7c18:b5ff:fe18:e024%13

==> k3s_daemon.log <==
Nov 28 05:18:28 truenas k3s[5796]: time="2022-11-28T05:18:28-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:18:29 truenas k3s[5796]: I1128 05:18:29.604754    5796 event.go:294] "Event occurred" object="ix-myinfinity/myinfinity-ix-chart" fieldPath="" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set myinfinity-ix-chart-66cfb9b6 to 1"
Nov 28 05:18:29 truenas k3s[5796]: I1128 05:18:29.653442    5796 event.go:294] "Event occurred" object="ix-myinfinity/myinfinity-ix-chart-66cfb9b6" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: myinfinity-ix-chart-66cfb9b6-xj2gb"
Nov 28 05:18:29 truenas k3s[5796]: I1128 05:18:29.737404    5796 topology_manager.go:205] "Topology Admit Handler"
Nov 28 05:18:29 truenas k3s[5796]: I1128 05:18:29.746966    5796 reconciler.go:357] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8tbb\" (UniqueName: \"kubernetes.io/projected/ce19492f-dcfe-4c07-9ee2-3a7298ab2027-kube-api-access-v8tbb\") pod \"myinfinity-ix-chart-66cfb9b6-xj2gb\" (UID: \"ce19492f-dcfe-4c07-9ee2-3a7298ab2027\") " pod="ix-myinfinity/myinfinity-ix-chart-66cfb9b6-xj2gb"
Nov 28 05:18:29 truenas k3s[5796]: I1128 05:18:29.747212    5796 reconciler.go:357] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ix-host-path-myinfinity-0\" (UniqueName: \"kubernetes.io/host-path/ce19492f-dcfe-4c07-9ee2-3a7298ab2027-ix-host-path-myinfinity-0\") pod \"myinfinity-ix-chart-66cfb9b6-xj2gb\" (UID: \"ce19492f-dcfe-4c07-9ee2-3a7298ab2027\") " pod="ix-myinfinity/myinfinity-ix-chart-66cfb9b6-xj2gb"

==> app_mounts.log <==
Nov 28 05:18:30 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-a55f08baa0274fff578a1049485d3227105a1ecd175aa3e46d875db94da19b12\x2dinit-merged.mount: Succeeded.
Nov 28 05:18:30 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-a55f08baa0274fff578a1049485d3227105a1ecd175aa3e46d875db94da19b12-merged.mount: Succeeded.
Nov 28 05:18:30 truenas systemd[1]: run-docker-runtime\x2drunc-moby-6529b26554a0103d57ec0f7af999ed2d2695fc26acf3f9f59662adf1150f036e-runc.uA9vBD.mount: Succeeded.

==> containerd.log <==
Nov 28 05:18:30 truenas dockerd[5679]: time="2022-11-28T05:18:30.551040699-08:00" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Nov 28 05:18:30 truenas dockerd[5679]: time="2022-11-28T05:18:30.551110184-08:00" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Nov 28 05:18:30 truenas dockerd[5679]: time="2022-11-28T05:18:30.551125092-08:00" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Nov 28 05:18:30 truenas dockerd[5679]: time="2022-11-28T05:18:30.551238438-08:00" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/6529b26554a0103d57ec0f7af999ed2d2695fc26acf3f9f59662adf1150f036e pid=18093 runtime=io.containerd.runc.v2

==> kern.log <==
Nov 28 05:18:31 truenas kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth66ab12c1: link becomes ready
Nov 28 05:18:31 truenas kernel: kube-bridge: port 4(veth66ab12c1) entered blocking state
Nov 28 05:18:31 truenas kernel: kube-bridge: port 4(veth66ab12c1) entered disabled state
Nov 28 05:18:31 truenas kernel: device veth66ab12c1 entered promiscuous mode
Nov 28 05:18:31 truenas kernel: kube-bridge: port 4(veth66ab12c1) entered blocking state
Nov 28 05:18:31 truenas kernel: kube-bridge: port 4(veth66ab12c1) entered forwarding state

==> k3s_daemon.log <==
Nov 28 05:18:30 truenas k3s[5796]: time="2022-11-28T05:18:30-08:00" level=info msg="Will attempt to re-write config file /mnt/tank/ix-applications/docker/containers/6529b26554a0103d57ec0f7af999ed2d2695fc26acf3f9f59662adf1150f036e/resolv.conf as [nameserver 192.168.1.1]"

==> daemon.log <==
Nov 28 05:18:30 truenas ntpd[2676]: Listen normally on 22 vethd5bc0ac5 [fe80::7c18:b5ff:fe18:e024%13]:123
Nov 28 05:18:31 truenas systemd-udevd[18012]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Nov 28 05:18:31 truenas systemd-udevd[18012]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.

==> app_mounts.log <==
Nov 28 05:18:31 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-8f9c0e8a96b67f3a7e3f5e48ef80c2e4275b9700e81f4a141c5e88bec4a656b7\x2dinit-merged.mount: Succeeded.

==> k3s_daemon.log <==
Nov 28 05:18:31 truenas k3s[5796]: I1128 05:18:31.212274   18146 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ix-myinfinity", Name:"myinfinity-ix-chart-66cfb9b6-xj2gb", UID:"ce19492f-dcfe-4c07-9ee2-3a7298ab2027", APIVersion:"v1", ResourceVersion:"98444", FieldPath:""}): type: 'Normal' reason: 'AddedInterface' Add eth0 [172.16.0.98/16] from ix-net

==> containerd.log <==
Nov 28 05:18:32 truenas dockerd[5679]: time="2022-11-28T05:18:32.613285261-08:00" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Nov 28 05:18:32 truenas dockerd[5679]: time="2022-11-28T05:18:32.613371333-08:00" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Nov 28 05:18:32 truenas dockerd[5679]: time="2022-11-28T05:18:32.613390294-08:00" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Nov 28 05:18:32 truenas dockerd[5679]: time="2022-11-28T05:18:32.613530362-08:00" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708 pid=18194 runtime=io.containerd.runc.v2

==> app_mounts.log <==
Nov 28 05:18:32 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-b013d5be29aaa1798ce15636322a6b7229d90d3a3e3afa474c3970f05b7fd3b9-merged.mount: Succeeded.
Nov 28 05:18:32 truenas systemd[1]: run-docker-runtime\x2drunc-moby-a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708-runc.xt2KG7.mount: Succeeded.

==> daemon.log <==
Nov 28 05:18:33 truenas ntpd[2676]: Listen normally on 23 veth66ab12c1 [fe80::a0fd:dcff:fe1b:8615%14]:123

==> k3s_daemon.log <==
Nov 28 05:18:33 truenas k3s[5796]: time="2022-11-28T05:18:33-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"

==> containerd.log <==
Nov 28 05:18:35 truenas dockerd[5679]: time="2022-11-28T05:18:35.919800543-08:00" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Nov 28 05:18:35 truenas dockerd[5679]: time="2022-11-28T05:18:35.919869764-08:00" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Nov 28 05:18:35 truenas dockerd[5679]: time="2022-11-28T05:18:35.919884325-08:00" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Nov 28 05:18:35 truenas dockerd[5679]: time="2022-11-28T05:18:35.920086349-08:00" level=info msg="starting signal loop" namespace=moby path=/run/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/6e4f049344faa2023bfe7f5c25851f65f41cf102823fadb7f782e0d39ceb0e34 pid=18440 runtime=io.containerd.runc.v2

==> app_mounts.log <==
Nov 28 05:18:35 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-8f9c0e8a96b67f3a7e3f5e48ef80c2e4275b9700e81f4a141c5e88bec4a656b7-merged.mount: Succeeded.
Nov 28 05:18:35 truenas systemd[1]: run-docker-runtime\x2drunc-moby-6e4f049344faa2023bfe7f5c25851f65f41cf102823fadb7f782e0d39ceb0e34-runc.zgNbtX.mount: Succeeded.

==> k3s_daemon.log <==
Nov 28 05:18:36 truenas k3s[5796]: I1128 05:18:36.209659    5796 event.go:294] "Event occurred" object="ix-myinfinity/myinfinity-ix-chart" fieldPath="" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled down replica set myinfinity-ix-chart-5f5578fdbb to 0 from 1"
Nov 28 05:18:36 truenas k3s[5796]: I1128 05:18:36.259661    5796 event.go:294] "Event occurred" object="ix-myinfinity/myinfinity-ix-chart-5f5578fdbb" fieldPath="" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulDelete" message="Deleted pod: myinfinity-ix-chart-5f5578fdbb-vwscc"
Nov 28 05:18:37 truenas k3s[5796]: {"level":"warn","ts":"2022-11-28T05:18:37.719-0800","logger":"etcd-client","caller":"v3@v3.5.3-k3s1/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0007108c0/kine.sock","attempt":0,"error":"rpc error: code = Unknown desc = no such table: dbstat"}
Nov 28 05:18:38 truenas k3s[5796]: time="2022-11-28T05:18:38-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"

==> kern.log <==
Nov 28 05:18:40 truenas kernel: kube-bridge: port 1(vethd5bc0ac5) entered disabled state
Nov 28 05:18:40 truenas kernel: device vethd5bc0ac5 left promiscuous mode
Nov 28 05:18:40 truenas kernel: kube-bridge: port 1(vethd5bc0ac5) entered disabled state

==> app_mounts.log <==
Nov 28 05:18:40 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-b013d5be29aaa1798ce15636322a6b7229d90d3a3e3afa474c3970f05b7fd3b9-merged.mount: Succeeded.

==> containerd.log <==
Nov 28 05:18:40 truenas dockerd[5674]: time="2022-11-28T05:18:40.428379242-08:00" level=info msg="ignoring event" container=a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Nov 28 05:18:40 truenas dockerd[5679]: time="2022-11-28T05:18:40.429892508-08:00" level=info msg="shim disconnected" id=a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708
Nov 28 05:18:40 truenas dockerd[5679]: time="2022-11-28T05:18:40.430161801-08:00" level=warning msg="cleaning up after shim disconnected" id=a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708 namespace=moby
Nov 28 05:18:40 truenas dockerd[5679]: time="2022-11-28T05:18:40.430240043-08:00" level=info msg="cleaning up dead shim"
Nov 28 05:18:40 truenas dockerd[5679]: time="2022-11-28T05:18:40.439730896-08:00" level=warning msg="cleanup warnings time=\"2022-11-28T05:18:40-08:00\" level=info msg=\"starting signal loop\" namespace=moby pid=18812 runtime=io.containerd.runc.v2\n"
Nov 28 05:18:40 truenas dockerd[5674]: time="2022-11-28T05:18:40.563590524-08:00" level=info msg="ignoring event" container=8c36961e13e44f42949c775dd4d55cb2a7cbf54148c888ff2b151c24c2f97539 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"

==> k3s_daemon.log <==
Nov 28 05:18:40 truenas k3s[5796]: I1128 05:18:40.720245    5796 reconciler.go:211] "operationExecutor.UnmountVolume started for volume \"ix-host-path-myinfinity-0\" (UniqueName: \"kubernetes.io/host-path/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb-ix-host-path-myinfinity-0\") pod \"4556faa0-5aaf-417e-9ccb-f801f1f3c5cb\" (UID: \"4556faa0-5aaf-417e-9ccb-f801f1f3c5cb\") "

==> app_mounts.log <==
Nov 28 05:18:40 truenas systemd[1]: run-docker-netns-0425388a3437.mount: Succeeded.
Nov 28 05:18:40 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-containers-8c36961e13e44f42949c775dd4d55cb2a7cbf54148c888ff2b151c24c2f97539-mounts-shm.mount: Succeeded.
Nov 28 05:18:40 truenas systemd[1]: mnt-tank-ix\x2dapplications-docker-overlay2-1da93e7f296f5a884b0b748809d82c257924a6d30c38abdd3e0e3774b6b15f70-merged.mount: Succeeded.

==> containerd.log <==
Nov 28 05:18:40 truenas dockerd[5679]: time="2022-11-28T05:18:40.564565567-08:00" level=info msg="shim disconnected" id=8c36961e13e44f42949c775dd4d55cb2a7cbf54148c888ff2b151c24c2f97539
Nov 28 05:18:40 truenas dockerd[5679]: time="2022-11-28T05:18:40.564687809-08:00" level=warning msg="cleaning up after shim disconnected" id=8c36961e13e44f42949c775dd4d55cb2a7cbf54148c888ff2b151c24c2f97539 namespace=moby
Nov 28 05:18:40 truenas dockerd[5679]: time="2022-11-28T05:18:40.564763593-08:00" level=info msg="cleaning up dead shim"
Nov 28 05:18:40 truenas dockerd[5679]: time="2022-11-28T05:18:40.607783522-08:00" level=warning msg="cleanup warnings time=\"2022-11-28T05:18:40-08:00\" level=info msg=\"starting signal loop\" namespace=moby pid=18864 runtime=io.containerd.runc.v2\n"

==> app_mounts.log <==
Nov 28 05:18:40 truenas systemd[1]: var-lib-kubelet-pods-4556faa0\x2d5aaf\x2d417e\x2d9ccb\x2df801f1f3c5cb-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2drrjzj.mount: Succeeded.

==> k3s_daemon.log <==
Nov 28 05:18:40 truenas k3s[5796]: I1128 05:18:40.732452    5796 reconciler.go:211] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrjzj\" (UniqueName: \"kubernetes.io/projected/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb-kube-api-access-rrjzj\") pod \"4556faa0-5aaf-417e-9ccb-f801f1f3c5cb\" (UID: \"4556faa0-5aaf-417e-9ccb-f801f1f3c5cb\") "
Nov 28 05:18:40 truenas k3s[5796]: I1128 05:18:40.728139    5796 operation_generator.go:890] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb-ix-host-path-myinfinity-0" (OuterVolumeSpecName: "ix-host-path-myinfinity-0") pod "4556faa0-5aaf-417e-9ccb-f801f1f3c5cb" (UID: "4556faa0-5aaf-417e-9ccb-f801f1f3c5cb"). InnerVolumeSpecName "ix-host-path-myinfinity-0". PluginName "kubernetes.io/host-path", VolumeGidValue ""
Nov 28 05:18:40 truenas k3s[5796]: I1128 05:18:40.745787    5796 operation_generator.go:890] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb-kube-api-access-rrjzj" (OuterVolumeSpecName: "kube-api-access-rrjzj") pod "4556faa0-5aaf-417e-9ccb-f801f1f3c5cb" (UID: "4556faa0-5aaf-417e-9ccb-f801f1f3c5cb"). InnerVolumeSpecName "kube-api-access-rrjzj". PluginName "kubernetes.io/projected", VolumeGidValue ""
Nov 28 05:18:40 truenas k3s[5796]: I1128 05:18:40.833931    5796 reconciler.go:399] "Volume detached for volume \"ix-host-path-myinfinity-0\" (UniqueName: \"kubernetes.io/host-path/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb-ix-host-path-myinfinity-0\") on node \"ix-truenas\" DevicePath \"\""
Nov 28 05:18:40 truenas k3s[5796]: I1128 05:18:40.834440    5796 reconciler.go:399] "Volume detached for volume \"kube-api-access-rrjzj\" (UniqueName: \"kubernetes.io/projected/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb-kube-api-access-rrjzj\") on node \"ix-truenas\" DevicePath \"\""
Nov 28 05:18:41 truenas k3s[5796]: I1128 05:18:41.251084    5796 scope.go:115] "RemoveContainer" containerID="a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708"
Nov 28 05:18:41 truenas k3s[5796]: I1128 05:18:41.355418    5796 scope.go:115] "RemoveContainer" containerID="a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708"
Nov 28 05:18:41 truenas k3s[5796]: E1128 05:18:41.358063    5796 remote_runtime.go:599] "ContainerStatus from runtime service failed" err="rpc error: code = Unknown desc = Error: No such container: a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708" containerID="a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708"
Nov 28 05:18:41 truenas k3s[5796]: I1128 05:18:41.358099    5796 pod_container_deletor.go:52] "DeleteContainer returned error" containerID={Type:docker ID:a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708} err="failed to get container status \"a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708\": rpc error: code = Unknown desc = Error: No such container: a3c70e10c576f3fb5ddd30fd1953069251684d3a83973f968cbbc798a3e47708"

==> daemon.log <==
Nov 28 05:18:41 truenas ntpd[2676]: Deleting interface #22 vethd5bc0ac5, fe80::7c18:b5ff:fe18:e024%13#123, interface stats: received=0, sent=0, dropped=0, active_time=11 secs

==> k3s_daemon.log <==
Nov 28 05:18:42 truenas k3s[5796]: I1128 05:18:42.414055    5796 kubelet_volumes.go:160] "Cleaned up orphaned pod volumes dir" podUID=4556faa0-5aaf-417e-9ccb-f801f1f3c5cb path="/var/lib/kubelet/pods/4556faa0-5aaf-417e-9ccb-f801f1f3c5cb/volumes"
Nov 28 05:18:43 truenas k3s[5796]: time="2022-11-28T05:18:43-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:18:48 truenas k3s[5796]: time="2022-11-28T05:18:48-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:18:53 truenas k3s[5796]: time="2022-11-28T05:18:53-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:18:58 truenas k3s[5796]: time="2022-11-28T05:18:58-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:19:03 truenas k3s[5796]: time="2022-11-28T05:19:03-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:19:08 truenas k3s[5796]: time="2022-11-28T05:19:08-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:19:13 truenas k3s[5796]: time="2022-11-28T05:19:13-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:19:19 truenas k3s[5796]: time="2022-11-28T05:19:19-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:19:20 truenas k3s[5796]: {"level":"warn","ts":"2022-11-28T05:19:20.191-0800","logger":"etcd-client","caller":"v3@v3.5.3-k3s1/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0007108c0/kine.sock","attempt":0,"error":"rpc error: code = Unknown desc = no such table: dbstat"}
Nov 28 05:19:24 truenas k3s[5796]: time="2022-11-28T05:19:24-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:19:29 truenas k3s[5796]: time="2022-11-28T05:19:29-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:19:34 truenas k3s[5796]: time="2022-11-28T05:19:34-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"
Nov 28 05:19:39 truenas k3s[5796]: time="2022-11-28T05:19:39-08:00" level=info msg="Using CNI configuration file /etc/cni/net.d/00-multus.conf"