Nextcloud Hanging on deployment 1/2 in Truenas Scale

omecabs

Cadet
Joined
Feb 24, 2021
Messages
3
Nextcloud Hanging on deployment ½ in Truenas Scale

Hello everyone, I'm relatively new to the Truenas Scale/NextCloud environment and have just installed Truenas Scale OS Version:TrueNAS-SCALE-22.12.2 on a

Product:ProLiant DL360 G6 Model:Intel(R) Xeon(R) CPU X5550 @ 2.67GHz Memory:94 GiB



I have been successful in deploying a couple of Official Charts apps however when I attempt to install NC 26.0.1_1.6.26 from the Official Charts repo I get inconsistent results. I have installed and removed it several times, rebooted the Truenas system as recommended by some, no luck.

My latest attempt results in the deployment hanging
1684252469835.png





The logs indicate an issue with nginx but I’m not sure why nor how to resolve. I’m not really versed in Docker/containers which may be the reason. Any assistance is greatly appreciated. Thanks in advance.

The configuration I’m using is as follows:
1684252499591.png



I have chosen an external dataset for the user files (which I will also expose via SMB) with the following permissions:

1684252520433.png




Host path location is:
1684252548387.png



The Host Path / application configuration files location is suggested by Tom Lawrence in his talk: https://youtu.be/vXGs221il3g

Having the following permissions:
1684252611362.png



Rest of Config:
1684252624570.png





When I look in the logs for: Application Name:nextcloud Pod Name:nextcloud-postgres-575d858bb-lbww4 Container Name:nextcloud-postgres


I see the following which shows that PostgreSQL version is 13.1:

2023-05-15 18:45:33.102079+00:002023-05-15T18:45:33.102079793Z

2023-05-15 18:45:33.102136+00:00PostgreSQL Database directory appears to contain a database; Skipping initialization

2023-05-15 18:45:33.102151+00:002023-05-15T18:45:33.102151182Z

2023-05-15 18:45:33.190512+00:002023-05-15 18:45:33.190 UTC [1] LOG: starting PostgreSQL 13.1 (Debian 13.1-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit

2023-05-15 18:45:33.190664+00:002023-05-15 18:45:33.190 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432

2023-05-15 18:45:33.190699+00:002023-05-15 18:45:33.190 UTC [1] LOG: listening on IPv6 address "::", port 5432

2023-05-15 18:45:33.242428+00:002023-05-15 18:45:33.242 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"

2023-05-15 18:45:33.304942+00:002023-05-15 18:45:33.304 UTC [26] LOG: database system was shut down at 2023-05-15 18:42:01 UTC

2023-05-15 18:45:33.340664+00:002023-05-15 18:45:33.340 UTC [1] LOG: database system is ready to accept connections



Logs for Application Name:nextcloud Pod Name:nextcloud-5c5bf8b544-m6wzc Container Name:nextcloud

2023-05-15 19:50:51.116666+00:00Conf remoteip disabled.

2023-05-15 19:50:51.116715+00:00To activate the new configuration, you need to run:

2023-05-15 19:50:51.116730+00:00service apache2 reload

2023-05-15 19:50:51.226886+00:00AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.16.0.119. Set the 'ServerName' directive globally to suppress this message

2023-05-15 19:50:51.250481+00:00AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.16.0.119. Set the 'ServerName' directive globally to suppress this message

2023-05-15 19:50:51.283782+00:00[Mon May 15 19:50:51.283632 2023] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.56 (Debian) PHP/8.1.19 configured -- resuming normal operations

2023-05-15 19:50:51.283830+00:00[Mon May 15 19:50:51.283689 2023] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'

2023-05-15 19:50:52.496995+00:00127.0.0.1 - - [15/May/2023:19:50:52 +0000] "GET /status.php HTTP/1.1" 503 3740 "-" "kube-probe/1.25"

2023-05-15 19:50:57.302877+00:00127.0.0.1 - - [15/May/2023:19:50:57 +0000] "GET /status.php HTTP/1.1" 503 3750 "-" "kube-probe/1.25"

2023-05-15 19:51:02.301937+00:00127.0.0.1 - - [15/May/2023:19:51:02 +0000] "GET /status.php HTTP/1.1" 503 3740 "-" "kube-probe/1.25"

2023-05-15 19:51:07.299500+00:00172.16.0.1 - - [15/May/2023:19:51:07 +0000] "GET /status.php HTTP/1.1" 503 3744 "-" "kube-probe/1.25"

2023-05-15 19:51:07.301824+00:00127.0.0.1 - - [15/May/2023:19:51:07 +0000] "GET /status.php HTTP/1.1" 503 3746 "-" "kube-probe/1.25"

2023-05-15 19:51:12.298674+00:00127.0.0.1 - - [15/May/2023:19:51:12 +0000] "GET /status.php HTTP/1.1" 503 3740 "-" "kube-probe/1.25"

2023-05-15 19:51:17.303626+00:00127.0.0.1 - - [15/May/2023:19:51:17 +0000] "GET /status.php HTTP/1.1" 503 3746 "-" "kube-probe/1.25"

2023-05-15 19:51:17.303691+00:00172.16.0.1 - - [15/May/2023:19:51:17 +0000] "GET /status.php HTTP/1.1" 503 3742 "-" "kube-probe/1.25"

2023-05-15 19:51:22.299135+00:00127.0.0.1 - - [15/May/2023:19:51:22 +0000] "GET /status.php HTTP/1.1" 503 3746 "-" "kube-probe/1.25"

2023-05-15 19:51:27.295067+00:00172.16.0.1 - - [15/May/2023:19:51:27 +0000] "GET /status.php HTTP/1.1" 503 3744 "-" "kube-probe/1.25"

2023-05-15 19:51:27.297018+00:00127.0.0.1 - - [15/May/2023:19:51:27 +0000] "GET /status.php HTTP/1.1" 503 3752 "-" "kube-probe/1.25"

2023-05-15 19:51:32.298276+00:00127.0.0.1 - - [15/May/2023:19:51:32 +0000] "GET /status.php HTTP/1.1" 503 3748 "-" "kube-probe/1.25"

2023-05-15 19:51:37.297133+00:00172.16.0.1 - - [15/May/2023:19:51:37 +0000] "GET /status.php HTTP/1.1" 503 3736 "-" "kube-probe/1.25"

2023-05-15 19:51:37.298719+00:00127.0.0.1 - - [15/May/2023:19:51:37 +0000] "GET /status.php HTTP/1.1" 503 3740 "-" "kube-probe/1.25"

2023-05-15 19:51:42.299250+00:00127.0.0.1 - - [15/May/2023:19:51:42 +0000] "GET /status.php HTTP/1.1" 503 3744 "-" "kube-probe/1.25"

2023-05-15 19:51:47.295589+00:00172.16.0.1 - - [15/May/2023:19:51:47 +0000] "GET /status.php HTTP/1.1" 503 3740 "-" "kube-probe/1.25"

2023-05-15 19:51:47.299512+00:00127.0.0.1 - - [15/May/2023:19:51:47 +0000] "GET /status.php HTTP/1.1" 503 3744 "-" "kube-probe/1.25"

2023-05-15 19:51:48.537052+00:00[Mon May 15 19:51:48.536917 2023] [mpm_prefork:notice] [pid 1] AH00170: caught SIGWINCH, shutting down gracefully



Log for Application Name:nextcloud Pod Name:nextcloud-5c5bf8b544-m6wzc Container Name:nginx

This seems to be the issue however I’ve not been able to figure out why and how to resolve.

2023-05-15 19:53:40.395329+00:00/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration

2023-05-15 19:53:40.395398+00:00/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/

2023-05-15 19:53:40.396912+00:00/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh

2023-05-15 19:53:43.079595+00:0010-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf

2023-05-15 19:53:43.085754+00:0010-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf

2023-05-15 19:53:43.085922+00:00/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh

2023-05-15 19:53:43.088614+00:00/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh

2023-05-15 19:53:43.090103+00:00/docker-entrypoint.sh: Configuration complete; ready for start up

2023-05-15 19:53:52.290005+00:002023/05/15 19:53:52 [error] 29#29: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://127.0.0.1:80/status.php", host: "localhost"

2023-05-15 19:53:52.290092+00:002023/05/15 19:53:52 [error] 29#29: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://127.0.0.1:80/status.php", host: "localhost"

2023-05-15 19:53:52.290038+00:00172.16.0.1 - - [15/May/2023:19:53:52 +0000] "GET /status.php HTTP/2.0" 502 157 "-" "kube-probe/1.25"

2023-05-15 19:53:57.289066+00:002023/05/15 19:53:57 [error] 29#29: *4 no live upstreams while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://localhost/status.php", host: "localhost"

2023-05-15 19:53:57.289112+00:00172.16.0.1 - - [15/May/2023:19:53:57 +0000] "GET /status.php HTTP/2.0" 502 157 "-" "kube-probe/1.25"

2023-05-15 19:54:02.289113+00:002023/05/15 19:54:02 [error] 29#29: *5 no live upstreams while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://localhost/status.php", host: "localhost"

2023-05-15 19:54:02.289130+00:00172.16.0.1 - - [15/May/2023:19:54:02 +0000] "GET /status.php HTTP/2.0" 502 157 "-" "kube-probe/1.25"

2023-05-15 19:54:07.288940+00:002023/05/15 19:54:07 [error] 29#29: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://127.0.0.1:80/status.php", host: "localhost"

2023-05-15 19:54:07.289016+00:002023/05/15 19:54:07 [error] 29#29: *6 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://127.0.0.1:80/status.php", host: "localhost"

2023-05-15 19:54:07.289047+00:00172.16.0.1 - - [15/May/2023:19:54:07 +0000] "GET /status.php HTTP/2.0" 502 157 "-" "kube-probe/1.25"

2023-05-15 19:54:12.288639+00:002023/05/15 19:54:12 [error] 29#29: *9 no live upstreams while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://localhost/status.php", host: "localhost"

2023-05-15 19:54:12.288669+00:00172.16.0.1 - - [15/May/2023:19:54:12 +0000] "GET /status.php HTTP/2.0" 502 157 "-" "kube-probe/1.25"

2023-05-15 19:54:17.288533+00:002023/05/15 19:54:17 [error] 29#29: *10 no live upstreams while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://localhost/status.php", host: "localhost"

2023-05-15 19:54:17.288575+00:00172.16.0.1 - - [15/May/2023:19:54:17 +0000] "GET /status.php HTTP/2.0" 502 157 "-" "kube-probe/1.25"

2023-05-15 19:54:22.288490+00:002023/05/15 19:54:22 [error] 29#29: *11 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://127.0.0.1:80/status.php", host: "localhost"

2023-05-15 19:54:22.288562+00:002023/05/15 19:54:22 [error] 29#29: *11 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.0.1, server: localhost, request: "GET /status.php HTTP/2.0", upstream: "http://127.0.0.1:80/status.php", host: "localhost"



Log for Application Name:nextcloud Pod Name:nextcloud-5c5bf8b544-m6wzc Container Name:init-postgresdb

2023-05-15 18:45:33.306743+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:33.307081+00:00waiting for postgres

2023-05-15 18:45:35.355830+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:35.356223+00:00waiting for postgres

2023-05-15 18:45:37.404958+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:37.405266+00:00waiting for postgres

2023-05-15 18:45:39.453252+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:39.453501+00:00waiting for postgres

2023-05-15 18:45:41.501006+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:41.501229+00:00waiting for postgres

2023-05-15 18:45:43.548984+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:43.549196+00:00waiting for postgres

2023-05-15 18:45:45.598211+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:45.598628+00:00waiting for postgres

2023-05-15 18:45:47.646752+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:47.647130+00:00waiting for postgres

2023-05-15 18:45:49.695218+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:49.695465+00:00waiting for postgres

2023-05-15 18:45:51.743920+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:51.744216+00:00waiting for postgres

2023-05-15 18:45:53.793800+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:53.794213+00:00waiting for postgres

2023-05-15 18:45:55.842676+00:00nextcloud-postgres:5432 - no response

2023-05-15 18:45:55.843762+00:00waiting for postgres

2023-05-15 18:45:57.888355+00:00nextcloud-postgres:5432 - accepting connections

Additionally... I Deleted the app and reinstalled with the default options (no changes whatsoever) and now I do get to a page however this is the message now.
Error

Error while trying to initialise the database: An exception occurred while executing a query: SQLSTATE[42P07]: Duplicate table: 7 ERROR: relation “oc_migrations” already exists

Create an admin account

Just not getting why? I see this message lots out there, but no real answers.

Thanks in advanced.
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,694
same problem, please support
Since we have this same issue replicated... I'd recommend one of you "report-a-bug" if you can help engineering diagnose and resolve.

@omecabs You seem to have all the necessary information and diagnostics.

Leave the NAS-ticket number here for everyone to track.
 

gu3miles

Cadet
Joined
May 30, 2023
Messages
4
same, i have a clean install (from an upgrade from core), tried to re-install nextcloud since my jails don't carry over, starting fresh with a new database, deployment is hanging. I tried to delete the app, restart, and try installing again, same thing. When I to go the nextcloud login it times out, and in shell I do not see the nextcloud files at the default location to look at the config file, I don't think the files were ever created.
 

gu3miles

Cadet
Joined
May 30, 2023
Messages
4
In my nextcloud logs I get:

/var/www/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable and that '/var/www/' is executable

I've tried to chown www-data:www-data in the shell, but it didn't fix anything
 

Krowvin

Explorer
Joined
Jul 24, 2014
Messages
60
Was this bug reported? If so i'd like to follow along, if not i'll report it.

I'm still having the same issue and the TrueCharts install appears to have been pulled. I would prefer to use the IX install and not fiddle with doing the docker compose myself.

I can verify it works if I leave most of the storage mount options blank. It appears as if it is a permission issue when settings the host paths? The other comments appears to collab that as well.

I've got my storage exposed via NFS and have it mounting to my truenas box using an init script in TrueNAS UI. I then want to mount that via hostPath.
 

gu3miles

Cadet
Joined
May 30, 2023
Messages
4
It is certainly a permissions problem. I gave up and started over, by deleting all of my datasets, and re-installing truenas scale. Then I rebuilt the datasets, and reinstalled.

I didn't have the issue. It installed and mostly worked.
 

Krowvin

Explorer
Joined
Jul 24, 2014
Messages
60
It installed and mostly worked.
I take it your nextcloud is mounted to the local machine datasets and not NFS?

And if so could I get a few screenshots of your app setup, specifically those storage options? And what permissions you gave your pools?

My running theory at the moment is that it is not properly setting the html/nextcloud data permissions in the docker file/compose. I haven't looked into it more beyond looking at the permissions of a working install vs what the nextcloud ix install does.

Edit: Hearing it worked for you the second time does also make me wonder if it's user error for you and I (i.e. does not need to be reported)
 

fastzombies

Explorer
Joined
Aug 11, 2022
Messages
57
The last comment was:
Please upgrade to 22.12.3.1 and report back if problem persists.

I'm trying that now and will see if it all blows up in my face or not.
 

fastzombies

Explorer
Joined
Aug 11, 2022
Messages
57
Upgrade went fine but install fails the same. I've updated the Jira ticket but it looks like it was closed without resolution.
 

RdJustRd

Cadet
Joined
Nov 5, 2020
Messages
2
FWIW:

I had same issue. Only fix I got to work was delete all apps and even the ix-applications dataset. Started over completely. AND this time I did not change the admin password or add any links to datasets. Let it boot up nextcloud once, worked first try, entered the web portal and then changed PW and added dataset.

I had worked on this for a solid day...only way that worked for me....
 

cserver

Cadet
Joined
May 11, 2021
Messages
5
FWIW:

I had same issue. Only fix I got to work was delete all apps and even the ix-applications dataset. Started over completely. AND this time I did not change the admin password or add any links to datasets. Let it boot up nextcloud once, worked first try, entered the web portal and then changed PW and added dataset.

I had worked on this for a solid day...only way that worked for me....
Darn - did this and still not working for me. Syncthing works fine.
 

fastzombies

Explorer
Joined
Aug 11, 2022
Messages
57
Today there was a version update to 27.0.0_1.6.32. I was able to successfully deploy. I really do not know what the final solution was but I did these things and it worked:

Create a docker user and group. Created the datasets nextcloud/data, postgres/data and postgres/backup on whatever pools you want. Apply the most open default ACLs avaialble. On the postgres datasets, add docker uer and group with rwx and default mask option checked.

I left the same ip as truenas. I tried a different ip and I could not ping it.

I wish I knew exactly what worked but I don't. Hopefully there is some clue in there that helps someone else.
 

sstruke

Dabbler
Joined
Feb 2, 2017
Messages
37
Today there was a version update to 27.0.0_1.6.32. I was able to successfully deploy. I really do not know what the final solution was but I did these things and it worked:

Create a docker user and group. Created the datasets nextcloud/data, postgres/data and postgres/backup on whatever pools you want. Apply the most open default ACLs avaialble. On the postgres datasets, add docker uer and group with rwx and default mask option checked.

I left the same ip as truenas. I tried a different ip and I could not ping it.

I wish I knew exactly what worked but I don't. Hopefully there is some clue in there that helps someone else.
Can you post some pictures of your setup for better understanding. Thank you in advance
 

fastzombies

Explorer
Joined
Aug 11, 2022
Messages
57
I will post some setup pics when I get a chance, though it will be like playing darts blindfolded as I really don't know what worked.
 

fastzombies

Explorer
Joined
Aug 11, 2022
Messages
57
Disclaimer: I do not know if anything I did here actually fixed anything. First create a docker user:

truenas-cred-user-docker-0.png


truenas-cred-user-docker-1.png
 

fastzombies

Explorer
Joined
Aug 11, 2022
Messages
57
Then I create a postgres/data and postgres/baclup datasets on my ssd pool. I edit ACL, using the most permissive default choice for both datasets. Then I add docker user and docker group.

NOTE: the docker groups appears to get created with the user, if you try to create a docker groups it tells you already exists.

truenas-dataset-nextclud-postgres-0.png


truenas-datasets-nextcloud-postgres-1.png
 

fastzombies

Explorer
Joined
Aug 11, 2022
Messages
57
Lastly I create a nextcloud/data dataset, again edit ACL with most permissive default choice. Then I add www-data user and group:

truehas-datasets-nextcloud-data-0.png


truenas-datasets-nextcloud-data-1.png
 
Top