Failed to upgrade chart: Error: pre-upgrade hook already exists

[barti04]

After a power outage during a Nextcloud update, the Chart is unable to start. When trying to update again, I get the following error:

[EFAULT] Failed to upgrade chart release: Error: UPGRADE FAILED: pre-upgrade hooks failed: warning: Hook pre-upgrade nextcloud/templates/nuke-deployments-hook.yaml failed: jobs.batch "pre-upgrade-hook1" already exists

I already tried a rollback but success, it still won't get past the “deploying phase“ and the hook error still persists.
There are also no logs loading for the chart.
What can I do?

Complete error:
Error: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/middlewared/job.py", line 411, in run await self.future File "/usr/lib/python3/dist-packages/middlewared/job.py", line 446, in __run_body rv = await self.method(*([self] + args)) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1265, in nf return await func(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1133, in nf res = await f(*args, **kwargs) File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/upgrade.py", line 116, in upgrade await self.upgrade_chart_release(job, release, options) File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/upgrade.py", line 299, in upgrade_chart_release await self.middleware.call('chart.release.helm_action', release_name, chart_path, config, 'upgrade') File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1344, in call return await self._call( File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1304, in _call return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args) File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1205, in run_in_executor return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs)) File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run result = self.fn(*self.args, **self.kwargs) File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/helm.py", line 44, in helm_action raise CallError(f'Failed to {tn_action} chart release: {stderr.decode()}') middlewared.service_exception.CallError: [EFAULT] Failed to upgrade chart release: Error: UPGRADE FAILED: pre-upgrade hooks failed: warning: Hook pre-upgrade nextcloud/templates/nuke-deployments-hook.yaml failed: jobs.batch "pre-upgrade-hook1" already exists

 

[Tormak]

Hi i have the same problem when i wan't to update the charts, have you find a solution?

Code:

Error: [EFAULT] Failed to upgrade chart release: Error: UPGRADE FAILED: pre-upgrade hooks failed: warning: Hook pre-upgrade nextcloud/templates/nuke-deployments-hook.yaml failed: jobs.batch "pre-upgrade-hook1" already exist

Thx.

 

[bolt]

Hello.
Having the same issue.
Tried to upgrade from 24.0.3_1.6.6 to 24.0.3_1.6.7 at the time the server/pool was quite overloaded and could not complete the request. I see there is a pre-upgrade-book1 pod running/pending/available (don't know what to call it) which is preventing the upgrades. I've tried reaplying multiple time. Was waiting for the next update and upgrading now to the newest 1.6.8 but I get the same error as above.
Any help would be appreciated.

Cheers

 

[bolt]

After a little bit of digging I think I got it... so answering my own question.

Not sure if the pods update to whatever version you were trying but be sure to be in one version (rollback) that works and allows upgrade after cleanup.

In my case I had a Pod lingering.
So in the truenas shell with root execute:

k3s kubectl get pods -A

identify you pre-upgrade-hook1 pod and go with

k3s kubectl delete pod <pod name> --namespace=ix-nextcloud

then locate the job with (we already know the name, just confirming it's there)

k3s kubectl get job --namespace=ix-nextcloud

and remove it

k3s kubectl delete job <job name> --namespace=ix-nextcloud

Make sure you use the namespace (this or another) otherwise it will use default and will not find anything.

Then proceed to upgrade the App and it should be ok.

Disclaimer: Use at your own risk. I take no responsibility if you loose your data.
I would be able to restore the data if anything went wrong. It worked for me does not mean it will work for you.

 

[Neillman]

I have this problem. When i try the solution, when i search the pod pre-upgrade-hook it doesn't exist in my list. How can i solve it?