kubelet pods in /mnt/var ?

[yottabit]

Code:

$ sudo find /mnt/var
/mnt/var
/mnt/var/lib
/mnt/var/lib/kubelet
/mnt/var/lib/kubelet/pods
/mnt/var/lib/kubelet/pods/f17bbbb4-3131-470e-b6cb-cd3c8a788932
/mnt/var/lib/kubelet/pods/f17bbbb4-3131-470e-b6cb-cd3c8a788932/volumes
/mnt/var/lib/kubelet/pods/f17bbbb4-3131-470e-b6cb-cd3c8a788932/volumes/kubernetes.io~csi
/mnt/var/lib/kubelet/pods/f17bbbb4-3131-470e-b6cb-cd3c8a788932/volumes/kubernetes.io~csi/pvc-cffef734-37a2-4633-95b0-a11ef3cdd2f2
/mnt/var/lib/kubelet/pods/f17bbbb4-3131-470e-b6cb-cd3c8a788932/volumes/kubernetes.io~csi/pvc-cffef734-37a2-4633-95b0-a11ef3cdd2f2/mount
/mnt/var/lib/kubelet/pods/5abc6a91-3ecf-42fb-afaa-3388c232a6cc
/mnt/var/lib/kubelet/pods/5abc6a91-3ecf-42fb-afaa-3388c232a6cc/volumes
/mnt/var/lib/kubelet/pods/5abc6a91-3ecf-42fb-afaa-3388c232a6cc/volumes/kubernetes.io~csi
/mnt/var/lib/kubelet/pods/5abc6a91-3ecf-42fb-afaa-3388c232a6cc/volumes/kubernetes.io~csi/pvc-a60047d2-a7fe-432a-ac5b-5bc8a3611145
/mnt/var/lib/kubelet/pods/5abc6a91-3ecf-42fb-afaa-3388c232a6cc/volumes/kubernetes.io~csi/pvc-a60047d2-a7fe-432a-ac5b-5bc8a3611145/mount
/mnt/var/lib/kubelet/pods/9ef4a2ee-7a7d-4db1-8255-3afd07f58772
/mnt/var/lib/kubelet/pods/9ef4a2ee-7a7d-4db1-8255-3afd07f58772/volumes
/mnt/var/lib/kubelet/pods/9ef4a2ee-7a7d-4db1-8255-3afd07f58772/volumes/kubernetes.io~csi
/mnt/var/lib/kubelet/pods/9ef4a2ee-7a7d-4db1-8255-3afd07f58772/volumes/kubernetes.io~csi/pvc-02554571-5688-4e41-a85f-cea1c610c502
/mnt/var/lib/kubelet/pods/9ef4a2ee-7a7d-4db1-8255-3afd07f58772/volumes/kubernetes.io~csi/pvc-02554571-5688-4e41-a85f-cea1c610c502/mount
/mnt/var/lib/kubelet/pods/dc80751d-e14d-47c9-a04d-7a673b5e8139
/mnt/var/lib/kubelet/pods/dc80751d-e14d-47c9-a04d-7a673b5e8139/volumes
/mnt/var/lib/kubelet/pods/dc80751d-e14d-47c9-a04d-7a673b5e8139/volumes/kubernetes.io~csi
/mnt/var/lib/kubelet/pods/dc80751d-e14d-47c9-a04d-7a673b5e8139/volumes/kubernetes.io~csi/pvc-02554571-5688-4e41-a85f-cea1c610c502
/mnt/var/lib/kubelet/pods/dc80751d-e14d-47c9-a04d-7a673b5e8139/volumes/kubernetes.io~csi/pvc-02554571-5688-4e41-a85f-cea1c610c502/mount

Code:

$ sudo zfs get mountpoint | grep pvc-
vol1/ix-applications/releases/mealie/volumes/pvc-41c5dff9-d331-4c74-8e4c-264b02e9da0d                        mountpoint  legacy                                                                                                                                   local
vol1/ix-applications/releases/syncthing/volumes/pvc-a60047d2-a7fe-432a-ac5b-5bc8a3611145                     mountpoint  /mnt/var/lib/kubelet/pods/5abc6a91-3ecf-42fb-afaa-3388c232a6cc/volumes/kubernetes.io~csi/pvc-a60047d2-a7fe-432a-ac5b-5bc8a3611145/mount  local

Any ideas what the other three are? Orphans? Safe to delete?

(And why does one of a mountpoint of 'legacy'? Take it up with Truecharts?)

 

[truecharts]

All ZFS backed PVC's have mountpoint legacy, thats an upstream requirement which both we and iX has no influence over.

(though we did, in collab with iX, test if it was possible to use a mountpath combined using datasets as PVC's, but we had to conclude it lead to issues, as could be expected with upstream's choices)

That being said:
Generally speaking you should not be deleting anything iX created at all.

 

[yottabit]

Thanks for the reply! But if all PVCs should have the legacy mountpoint, why does syncthing not?

Also, why are these two showing up in /mnt/var at all? That is not a mountpoint on my system, so it's part of the rootfs, and hence wouldn't its state be lost on reboot?

I see a bunch of kubelet/pods mounted as tmpfs in /var/lib, which makes more sense than /mnt/var/lib ...

And for pvc mounts specifically:

Code:

$ sudo findmnt -o TARGET,SOURCE,FSTYPE | grep pvc-
├─/var/lib/kubelet/pods/7eabece9-7f56-4d2f-a25f-00b6b82a487d/volumes/kubernetes.io~csi/pvc-41c5dff9-d331-4c74-8e4c-264b02e9da0d/mount vol1/ix-applications/releases/mealie/volumes/pvc-41c5dff9-d331-4c74-8e4c-264b02e9da0d                                                                                                  zfs

So it looks like mealie is valid with a legacy mountpoint; but what about syncthing? Why does it have a non-legacy mountpoint? And why is it present in /mnt/var?

 

[majinjing3]

I think there is something wrong in truenas scale.
All PVCs in my truenas scale are not legacy mountpoint.

 

[appliance]

same problem in Scale.. first, RAID was forced into /mnt/mnt.. Then another disk. Then kubelets into /mnt/var. Always something adds "/mnt" in front of paths. It just keeps coming.