SOLVED Nextcloud internal server error after php and version update

[russtysandwiches]

Hi All,

I'm a little stuck here. I've just updated from Nextcloud 27 to 28. I guessed that I'd need a php update as well if it keeps prompting me to update. So I ran
pkg install nextcloud-php83. The web UI then happily ran the Nextcloud version update without issue. However, about an hour later I browsed to the web UI and it gave me the ever-helpful "internal server error" with no Nextcloud UI elements. After some googling, I was informed to check the logs but when I headed to the log location I couldn't make any sense of it. Now I'm not exactly a pro at this, so I may have missed something.

Nextcloud is running as the official jail/plugin from Truenas core.
server specs (don't judge):
Xeon(R) CPU E3-1220 v3
16gb ddr3 ECC
4x 2TB HDD Raidz1 with 1 128GB SSD cache
10gbe fibre card for networking


Now in my googling, I've seen a few people suggest that the official plugin isn't great that I should have installed Nextcloud differently and that I should just do that instead. I'd rather not do that if possible. I've been using Nextcloud as a replacement for Onedrive/Google Drive and it has a lot in it. I'm also not savvy enough to point the new version at the older data.

Here's hoping for a magic silver line of code!

 

[victort]

What version of php are you running?
It will probably have to be removed first.

You can do that by pkg remove nextcloud-phpxx x is the php version.
Then try restarting the plugin.

 

[russtysandwiches]

What version of php are you running?

8.3

It will probably have to be removed first.

You can do that by pkg remove nextcloud-phpxx x is the php version.
Then try restarting the plugin.

just ran that for versions 8.0-.2.

pkg remove nextcloud-php80 No packages matched for pattern 'nextcloud-php80'

After a reboot still no joy.

Should I downgrade to 8.2 instead?

 

[Apollo]

8.3

just ran that for versions 8.0-.2.

pkg remove nextcloud-php80 No packages matched for pattern 'nextcloud-php80'

After a reboot still no joy.

Should I downgrade to 8.2 instead?

If you have snapshots of your iocage/jail/nextcloud/root... or similar, you can rollback to the most recent one prior to the upgrade.
When doing an update of Nextcloud from the web interface, it will create a backup folder of the previous install, so it is safe enough to rename the newly created Netcloud folder to keep aside and move/rename the backup with the nextcloud name.

PS: In the recent years, I have also seen the "internal server error" message after the update claimed to have worked properly, to find out the database was still in maintenance mode. This can be one of the issue. Another one could be related to the cache used with Nextcloud witch need to be properly configured.

 

[russtysandwiches]

If you have snapshots of your iocage/jail/nextcloud/root... or similar, you can rollback to the most recent one prior to the upgrade.
When doing an update of Nextcloud from the web interface, it will create a backup folder of the previous install, so it is safe enough to rename the newly created Netcloud folder to keep aside and move/rename the backup with the nextcloud name.

I don't have a snapshot sadly(I will google how to later) However I did do the update via the next cloud webui. How would one rollback?

PS: In the recent years, I have also seen the "internal server error" message after the update claimed to have worked properly, to find out the database was still in maintenance mode. This can be one of the issue. Another one could be related to the cache used with Nextcloud witch need to be properly configured.

Just checked the config.php file under /usr/local/www/nextcloud/config Maintenance mode is off.

 

[Patrick M. Hausen]

Nextcloud is running as the official jail/plugin from Truenas core.

If you run the plugin you are stuck with the plugin. You need to wait for the plugin maintainer to update the plugin for a current PHP version.

If you want current software, use a standard jail and manual install or the script provided by @danb35.

 

[victort]

If you need help setting the paths to your current files, I could help you do that. It's fairly straight forward.
How much space does the plugin currently use?

 

[russtysandwiches]

If you run the plugin you are stuck with the plugin. You need to wait for the plugin maintainer to update the plugin for a current PHP version.

so downgrade php to 8.1? It seems odd that the nextcloud updater mentioned I could update. Should I just ignore that in the future and just periodically update the plugin via truenas?

If you want current software, use a standard jail and manual install or the script provided by @danb35.

I may switch to this method later it seems the plugin has issues updating. Although perhaps they should be merged?

 

[victort]

so downgrade php to 8.1? It seems odd that the nextcloud updater mentioned I could update. Should I just ignore that in the future and just periodically update the plugin via truenas?

I may switch to this method later it seems the plugin has issues updating. Although perhaps they should be merged?

Unfortunately, if merging them were possible, it would have already been done. Plugins are very different from the scripts mentioned above.

Plugins are dependent on the maintainer.

The script installs nextcloud, and from there you take over. You can update to your hearts content. The script give you a working install from the get go, and does not require you to depend on it for updates as do the plugins.

 

[Apollo]

I don't have a snapshot sadly(I will google how to later) However I did do the update via the next cloud webui. How would one rollback?


Just checked the config.php file under /usr/local/www/nextcloud/config Maintenance mode is off.

Are you using the plugin or iocage jail? I was a bit confused with that statement?
Even though you may not be aware of existing snapshot, you might have some automatic snapshot task in place.
You could check for snapshot with the following CLI command:

zfs list -t snapshot -r tank/iocage/jails/nextcloud/root

Replace in the above line "tank" with the name of your pool that contain the jail amd "nextcloud" with the name of the plugin or jail.
You can get the name of the plugins/jails with:

iocage list

 

[Patrick M. Hausen]

so downgrade php to 8.1?

Roll back to last snapshot or delete and reinstall. Plugin means only plugin, don't try to manipulate anything in there manually.

 

[russtysandwiches]

If you need help setting the paths to your current files, I could help you do that. It's fairly straight forward.
How much space does the plugin currently use?

59.84 GiB My current plan is to patch this install together and then see if I can move my files to a different (more stable) instance of nextcloud, Probably running in there a jail using that script or in a full VM of its own. I am not sure that would be the best given my limited hardware but I'm not sure yet. Either way, id like a failover nextcloud instance (or snapshot). This isn't the first time I've had issues updating (mostly maintenance mode staying on or the ./htaccess issue). I've managed to google my way around those but it's rather frustrating.

 

[russtysandwiches]

Roll back to last snapshot or delete and reinstall. Plugin means only plugin, don't try to manipulate anything in there manually.

If I was to delete the plugin and reinstall, Would it keep the data?

 

[Apollo]

Roll back to last snapshot or delete and reinstall. Plugin means only plugin, don't try to manipulate anything in there manually.

59.84 GiB My current plan is to patch this install together and then see if I can move my files to a different (more stable) instance of nextcloud, Probably running in there a jail using that script or in a full VM of its own. I am not sure that would be the best given my limited hardware but I'm not sure yet. Either way, id like a failover nextcloud instance (or snapshot). This isn't the first time I've had issues updating (mostly maintenance mode staying on or the ./htaccess issue). I've managed to google my way around those but it's rather frustrating.

One question I have in the back of my head is how the data structure of the Nextcloud data and database? Are they embedded within the plugins path or do they exist as a share to an dedicated dataset outside the plugin/jail?
Roolback if possible could wipe existing Nextcloud user data and database after the snapshot being rollbacked from.

 

[russtysandwiches]

Well, I think I've found the relevant snapshots. Under the snapshots tab (funny that!) in the truenas webgui. I have begun the rollback to the last known good state (About a month ago). It seems mad at me for rolling back further than the most resent snapshot. In this case can I just turn off the safety checks?

 

[Apollo]

Don't rollback yet. You might loose your data from the past month.
Instead, we could look at replication to create a new version of your nextcloud plugin.

 

[Apollo]

I would go baby steps with the procedure in order to mitigate disaster.
But there a re a few questions that needs answering:
Are you the only one using Nectloud (ie one user) or do you share Nextcloud with other users having its own account in Nectloud?
Are your Nextcloud user data embedded in the plugin/jail or are they mounted into the plugin/jail from an external dataset?

 

[russtysandwiches]

I would go baby steps with the procedure in order to mitigate disaster.
But there a re a few questions that needs answering:
Are you the only one using Nectloud (ie one user) or do you share Nextcloud with other users having its own account in Nectloud?
Are your Nextcloud user data embedded in the plugin/jail or are they mounted into the plugin/jail from an external dataset?

its only me who uses this at the moment mainly as a onedrive/googledrive replacement.

I don't do much with it other than send all my photos, etc to it. I still have my Google accounts etc. Id love to push this to the rest of my fam but I don't trust my skill level just yet.
I slightly jumped the gun and rolled back to a known good state. Looks like I'm back to ver 27.0.2 Its let me login and its asking for an update. Should I?

How do I avoid this happening again?

Should I only update via the truenas plugin?

 

[Patrick M. Hausen]

Should I only update via the truenas plugin?

Yes, absolutely. If you use the plugin, only update via the plugin.

That's why most regulars here recommend using a manual/scripted install instead.

 

[Apollo]

its only me who uses this at the moment mainly as a onedrive/googledrive replacement.

I don't do much with it other than send all my photos, etc to it. I still have my Google accounts etc. Id love to push this to the rest of my fam but I don't trust my skill level just yet.
I slightly jumped the gun and rolled back to a known good state. Looks like I'm back to ver 27.0.2 Its let me login and its asking for an update. Should I?

How do I avoid this happening again?

Should I only update via the truenas plugin?

I use iocage/jail to host my Nextcloud instances (original script from @danb35 from a couple of years ago), unfortunately the newer Cloudflare option didn't work for me due to the DNS name provider not being part of the list of supported provider.
Regardless, as you have just experienced, Snapshots are very convenient to have, so next time you proceed with the update, do a snapshot first (ideally when the plugin/jail has been stopped preferably).
Having Nexcloud with a single account for yourself is ideal as you have precise control of the type of data you are hosting. With multiple user accounts for other people, you need to take into account the risk you bring by doing a rollback as it could delete data you have no knowledge about. The process remain the same but with an emphasis on data retention.

Because the data is yours (single user), you can experiment with the upgrade strategy more freely, but you need to have a backup plan so you don't loose your data. Replication is one way to do that.

Upgrading Nextcloud is not without risk, so having a snapshot to roll back from is key.
With a bit more freedom, you could replicate the plugin/jail and experiment with it as a fully independant jail, but this is not without any issues.

There are a few ways you can approach the problem and each one as its own benefit/issues.