Copy directories in to pod pv storage

[Ixian]

Kubectl's syntax for cp is different and I'm trying to figure out the correct way to copy local files and directories in to PVC storage.

For example, say I have configuration data for Radarr in /mnt/ssd-storage/appdata/radarr and I want to copy all the files and directories inside it to the /config directory my Radarr pod, where I have configured a pv. If I do:

Code:

k3s kubectl cp /mnt/ssd-storage/appdata/radarr ix-radarr/radarr-5c7cb4d676-xhj4g:/config

(where ix-radarr/radarr-5c7cb4d676-xhj4g is my persistent volume)

It copies everything to a new config/ sub-directory inside the /config directory, not the files and directories to /config itself.

Kubectl cp has different syntax than regular Unix-style cp (for example, recursive is implied) and it could be I'm missing something on how absolute/relative paths are handled but it eludes me. Obviously I can just open a shell to the app and use cp inside it to put the files in the correct structure but isn't there a better way to do this?

 

[FrostyCat]

The kubectl cp is not very powerful, you can either copy your files one by one (in a loop) or if the container happens to have useful binaries such as tar you can use those to package up your source and copy into the container to be unpackaged.

Example:

Code:

tar cf -C /some/dir . | k3s kubectl exec -ti -n <NS> <POD> [-c <CONTAINER>] -- tar xf - -C /some/target/dir

Occasionally you may get errors about permissions, timestamps, etc, so always double check if your files have been copied over.

Alternatively, and what I like to do is just mount the extra filesystem as some /path, exec into the pod, copy whatever I want then umount. Generally you will either do this by hand on an existing Deployment or DaemonSet or via the UI. Your pod will be recreated if you do this.

 

[Ixian]

That's not the issue I'm having. kubectl cp will copy multiple files and subdirectories, you don't need to loop them or tar them up. The recursive is implied, you don't need -r.

The issue is with how it deals with relative/absolute paths. Right now, when I attempt to copy the contents of my local ../radarr directory to the /config directory inside the pod it copies everything (including subdirectories) but to a sub-directory called config inside the config directory itself.

It handles paths differently than, say, SCP, and no amount of trying different combinations of /, etc. seems to work.

I can just copy it this way and then move stuff inside the pod via shell, sure, but I was wondering if I was missing something obvious and there's a way to have it copy to the right paths in the first place.

 

[truecharts]

We have a couple stickies on our discord with some different solutions besides kubectl exec and kubectl cp.
Generally we don't like those exec/cp based solutions at all, to be quite frank.

 

[Ixian]

We have a couple stickies on our discord with some different solutions besides kubectl exec and kubectl cp.
Generally we don't like those exec/cp based solutions at all, to be quite frank.

I don't disagree that it's clunky but as this exits RC I expect a lot of users are going to want to know how to copy data from existing docker, jail, etc. setups in to their new apps.

Bind mounts to the existing directories is one obvious way but those won't work with the app rollback feature unless I am mistaken in how it works (which is certainly possible).

 

[FrostyCat]

That's not the issue I'm having. kubectl cp will copy multiple files and subdirectories, you don't need to loop them or tar them up. The recursive is implied, you don't need -r.

The issue is with how it deals with relative/absolute paths. Right now, when I attempt to copy the contents of my local ../radarr directory to the /config directory inside the pod it copies everything (including subdirectories) but to a sub-directory called config inside the config directory itself.

I only mentioned the loop in case you want to not have the subdirectory created automatically.

In your case, since you have the files on the host, I would just skip the PVC and mount /config as hostPath, this way you'll keep the data even if you delete and redeploy the app. There are other solutions as well, but this one should be the easiest.

Just in case I'm not on the right path, what are you trying to achieve exactly?

 

[truecharts]

I don't disagree that it's clunky but as this exits RC I expect a lot of users are going to want to know how to copy data from existing docker, jail, etc. setups in to their new apps.

Bind mounts to the existing directories is one obvious way but those won't work with the app rollback feature unless I am mistaken in how it works (which is certainly possible).

As iX it not using PVC for their official Apps, It's best asked towards the catalog(s) that decided to use PVC storage...

We still advice the other solutions, as they give a little more feedback and room for error than kubectl exec/cp

 

[Ixian]

I only mentioned the loop in case you want to not have the subdirectory created automatically.

In your case, since you have the files on the host, I would just skip the PVC and mount /config as hostPath, this way you'll keep the data even if you delete and redeploy the app. There are other solutions as well, but this one should be the easiest.

Just in case I'm not on the right path, what are you trying to achieve exactly?

I am testing out the PVC storage backend that Truecharts has implemented for their apps - more out of curiosity and to learn how it works than anything else.

The use case is a user who has pre-existing persistant data from a prior docker/jail setup and wants to utilize PVC storage with the new apps. I already have hostpaths working, I was trying to find an efficient way to migrate data to PVC next since Truecharts has implemented a rollback feature in their apps.

If the answer is just to use hostpaths to existing datasets and snapshot/restore/etc. from there I am fine with that, that is simple.

 

[Ixian]

As iX it not using PVC for their official Apps, It's best asked towards the catalog(s) that decided to use PVC storage...

We still advice the other solutions, as they give a little more feedback and room for error than kubectl exec/cp

I found some stickied posts that I think you are referring to on Discord in the support thread, the ones about stopping the app, mounting the PVC locally, etc. if that is what you mean by other solutions.

It seems like, if you have existing persistant data in zfs datasets for apps, then just mount them as hostpaths ala bind mounts in docker and call it a day.

 

[FrostyCat]

It seems like, if you have existing persistant data in zfs datasets for apps, then just mount them as hostpaths ala bind mounts in docker and call it a day.

Yeah PVC is not always the answer, especially if you want to preserve and backup the data, as deleting an app will destroy the namespace and since PVCs are namespace bound resources they will be gone as well.

 

[truecharts]

It seems like, if you have existing persistant data in zfs datasets for apps, then just mount them as hostpaths ala bind mounts in docker and call it a day.

When we deploy something as PVC by default, we often do so for a reason. Changing that to hostPath is not a great idea as it breaks rollback.
Often when we deploy something as PVC, it's crucial that said data is reverted as well when doing a rollback. Rollbacks are also a very important part of the design, as we expect some updates to not work perfectly for all users, which means they depend on rollback to revert after an update.

Yeah PVC is not always the answer, especially if you want to preserve and backup the data, as deleting an app will destroy the namespace and since PVCs are namespace bound resources they will be gone as well.

We've covered that issue in our WIP backup guide :)

 

[FrostyCat]

Good to know, thanks!

 

[Ixian]

When we deploy something as PVC by default, we often do so for a reason. Changing that to hostPath is not a great idea as it breaks rollback.
Often when we deploy something as PVC, it's crucial that said data is reverted as well when doing a rollback. Rollbacks are also a very important part of the design, as we expect some updates to not work perfectly for all users, which means they depend on rollback to revert after an update.



We've covered that issue in our WIP backup guide :)

I assume you are talking about creating recursive snapshots of the specific ix-applications /releases dataset to manage the persistent data.

By "default" do you mean if the app configuration lists pvc as a default option in the storage section, or something else?