Deleted duplicate Nextcloud plugin, lost the stored data...restore from a snapshot?

[zdude]

Relatively new user, only set up this TrueNAS server about 8 months ago.

AMD Ryzen5 5600G
16GB RAM
256GB SSD system drive
2x4TB mirror pool (pool name = secondpool)
a 3rd 4TB is connected, but not in any pool; it held the original data which was imported into the TrueNAS pool. Can now be used to copy/backup any needed files, pools, data, etc., if needed, or added to the pool, if that's of benefit.

Installed Nextcloud plugin months ago, and have been auto uploading all pic/vids from my mobile phone (including the years of pics/vids which were already on the phone. All was well.

For some reason, I ended up with 2 nextcloud plugins, along with 2 jails for those plugins. I believe this occurred when I was trying to figure out the best way to duplicate/back up the entire nextcloud installation and its data. Those 2 instances coexisted peacefully for a few months, before I decided to mess with it more; I know, a very ignorant move to only partially understand TrueNAS Core and IOCage/jails prior to trying to manipulate the data. On 12/30, I uninstalled what I thought was the duplicate Nextcloud plugin, and both plugins and both jails were gone.

I have recursive snapshots of my only zfs pool (but no separate snapshot for the individual IOCage for Nextcloud), which were taken 1 or 2 days prior to losing the Nextcloud plugins/jails.

I understand cloning a dataset and mounting it as a share for indiv file recovery; I have already done that successfully.
What I don't understand, is whether it is possible, and how best I might try to recover my lost Nextcloud data using the snapshots of the pool.

I saw a snapshot named 'secondpool' auto-2022-12-28_00-00, and I rolled back to that, however, I didn't see the plugins or jails restored. As you can see in the screenshot of my snapshots, there are still 2 snapshots remaining which match up the timestamp 'auto-2022-12-28_00-00'. Not sure if I now just need to do something wth those, or install the plugin again first, etc.

I also have a full set of snapshots from the next day 12/29, with times auto-2022-12-29_14-59. Since I have a full set of these snapshots, maybe these are the ones to use?

Thanks for any help or direction that can be provided.

 

[sretalla]

I have recursive snapshots of my only zfs pool (but no separate snapshot for the individual IOCage for Nextcloud)

A recursive snapshot is effectively one snapshot for each child and one for the parent dataset, so you have what you need to put only the one the jail back.

Just roll back the snapshots related to that one jail (there should be at least 2 of them... nextcloud and nextcloud/root)

 

[zdude]

Thanks sretalla for the quick response,
This is where think something is wrong, as there are no snapshots shown which are further down the jail path; only secondpool/iocage/jails.
I dont see anything beyond that path point, nor anything that references nextcloud. Any ideas?

 

[sretalla]

This is where think something is wrong, as there are no snapshots shown which are further down the jail path; only secondpool/iocage/jails.
I dont see anything beyond that path point, nor anything that references nextcloud. Any ideas?

It looked to me like you had a scroll bar on the right of that list last time (your previous post).

Can you filter the snapshots by nextcloud ?

 

[zdude]

Yes, I do have many and there is a scroll bar and additional pages. I had filtered before, but found no nextcloud matches.
Am I SOL??

 

[sretalla]

you can try iocage snaplist

But in both cases, if you can't remember the name of the jail, you won't be able to find it (seems nextcloud may not be it?).

Another point that comes to mind...

You can't destroy a dataset which still has snapshots, so at some point in order for that dataset to be destroyed, the snapshots should also be gone. Depends how you're replicating to the second pool as to how likely it may be to have survived there if gone from the primary.

 

[winnielinnie]

If the dataset secondpool/iocage/jails/nextcloud is gone, then there is no way to restore any previous snapshots.

You can't destroy a dataset which still has snapshots, so at some point in order for that dataset to be destroyed, the snapshots should also be gone.

I haven't deleted a jail/plugin in a while, but I believe as long as you confirm "Yes", it will go ahead and forcefully destroy the associated dataset, and hence all of its snapshots, under poolname/iocage/jails/jailname

 

[sretalla]

But if that's replicated to another pool? Not sure of the details of the replication, so maybe also replicated the deletion.

 

[winnielinnie]

But if that's replicated to another pool? Not sure of the details of the replication, so maybe also replicated the deletion.

If it was previously replicated, and the dataset/snapshots exists on the other pool, then @zdude can replicated it back to this pool. However, I'm not sure if it will confuse the iocage system, since the jail itself was outright deleted. I never tried to "import" a jail from an existing dataset.

 

[zdude]

sretalla,
I wasn't doing any replication (unless there's some enabled by default).

Results of "iocage snaplist nextcloud":
jail 'nextcloud' not found


winnielinnie,
Thanks for your input. It's correct that the dataset secondpool/iocage/jails/nextcloud does not exist.
I dont understand how the snapshot didn't/doesn't exist, since I did not uninstall the plugin until afternoon of 12/30. Are you saying that it went back and deleted any snapshots related to the plugin which is being uninstalled?

Here's some clones of the datasets, which I mounted to look around and see if I could find any remnants of the nextcloud jail or file system anywhere, but so far no luck.
The 4tb-2 is my main dataset which is for smb sharing, and consists mainly of archives of my music and movies.

Looks like I'm running into a lot of dead-ends, and may just have to accept that I lost that ~25GB dataset.
Thankfully, I should have most, if not all, of the lost data in other places, scattered across numerous pc backups and on some cloud storages; I just need to set time aside to go through all of them again.

I know I need to devise a solid backup/replication system and get it in place, but aside from that: in order to avoid this type of loss in the future, it's advisable to store the plugin's dataset outside of the plugin's jail?

 

[winnielinnie]

Are you saying that it went back and deleted any snapshots related to the plugin which is being uninstalled?

When you delete a plugin or jail using the GUI or iocage command, it destroys the associated dataset, and hence its snapshots.

 

[winnielinnie]

I just tried a test to see what type of warnings the TrueNAS GUI gives you, and it appears it does not thoroughly explain what will be destroyed.

Using the GUI, I created a new jail named testjail

Using the GUI, I created recursive snapshots for poolname/iocage/jails/testjail

Using the GUI, I went to Jails -> testjail -> Delete

I was simply asked to confirm that I wanted to delete this jail. I clicked yes.

The datasets (and all associated snapshots) poolname/iocage/jails/testjail and poolname/iocage/jails/testjail/root were destroyed.


So it seems that you're not really given a thorough warning of what will happen if you delete the jail or uninstall a plugin.

 

[winnielinnie]

and may just have to accept that I lost that ~25GB dataset.

25 GB? Were you saving files directly within the jail's dataset itself? You didn't create mountpoints to your primary datasets that live outside of the iocage path?

 

[zdude]

25 GB? Were you saving files directly within the jail's dataset itself? You didn't create mountpoints to your primary datasets that live outside of the iocage path?

Yes, about 25GB. I actually think it is a bit lower for my data, but I recall seeing that the iocage was using about 25GB at some point recently.
I don't recall creating a mountpoint on my primary dataset (4tb-2). By default, doesn't it create a mountpoint in the jail? When I installed it, I didn't know enough about that to modify it.

 

[victort]

Just use @danb35 script here for your next install.

https://www.truenas.com/community/resources/scripted-installation-of-nextcloud-23-in-iocage-jail.90/

This way even if the jail is destroyed you still have your data.

 

[victort]

Just use @danb35 script here for your next install.

https://www.truenas.com/community/resources/scripted-installation-of-nextcloud-23-in-iocage-jail.90/

This way even if the jail is destroyed you still have your data.

I had all the trouble you had when I got started with Nextcloud until I got this script…

 

[zdude]

Just use @danb35 script here for your next install.

https://www.truenas.com/community/resources/scripted-installation-of-nextcloud-23-in-iocage-jail.90/

This way even if the jail is destroyed you still have your data.

Thanks, I will check that out, for sure!

 

[danb35]

Here's the fundamental problem: a jail (at least in Free/TrueNAS) is contained in a dataset. When you destroy the jail, you destroy that dataset. Snapshots belong to the dataset. So when you destroy a dataset, you destroy all its snapshots as well. If you replicated them somewhere else (i.e., to another pool, or to another system), you may be able to get them back, but otherwise, they're gone.

 

[winnielinnie]

By default, doesn't it create a mountpoint in the jail? When I installed it, I didn't know enough about that to modify it.

Whether or not it automatically creates a mountpoint, you need to save your files only into the mountpoint's path.

So for example, the path /media/data/ in your jail's filesystem hierarchy points to /mnt/mypool/nextcloud-data/, which reflects the dataset mypool/nextcloud-data. Anything saved here with (i.e, /media/data/) with the active jail will be saved outside of your iocage dataset structure.

Otherwise, you will end up saving your files to a path that resides within your iocage/jail's dataset, and hence is at risk for being lost if you were to ever uninstall a plugin/jail or redo your iocage setup.

 

[zdude]

danb35,
thanks for the great explanation. It's GONE.


One final snapshot question, before I go off and start my rebuilding of Nextcloud...

I did purposely delete many movies off my main pool dataset (4tb-2), which I thought were duplicates, to free up some space. Turns out some were not dupes, so I want to put them back. I began by mounting a dataset clone, enabling the share, and then dragging them back to the main dataset folder. It works, but is painfully slow, and I prob have close to 500GB-1TB to copy over.

I think it might be easier to do a rollback to that dataset of my earliest complete snapshots (obviously does not include the Nextcloud jail). Is the process that I just rollback the main pool snapshot, or is it better to just rollback the 4tb-2 dataset? Can I promote the cloned dataset, see if all looks good, and then delete the original 4tb-2 dataset?