Scripted installation of Nextcloud 28 in iocage jail 2018-03-23

[Ericloewe]

Via the magic of XenForo plugins, I have reassigned this thread to be the discussion thread for the new Resource.

 

[dureal99d]

It seems like there are lot of guides here on installing Nextcloud, and a number of people reporting problems with many of them. So, I figured, why not write another one? This is inspired by the work of @kjake in his scripts to build jails for Unifi and Crashplan, and largely follows the process that @dureal99d documented here. The difference between this method and the others is that this method employs a script to create the jail, install all the relevant packages, and configure them. My thought/hope is that by scripting the installation and setup, a lot of the PEBCAK-type errors that come up in such a complex installation can be avoided.

This script uses acme.sh to get a certificate for your jail. In order to obtain the cert, one of two things must be true:

1. You have a fully-qualified domain name ("$FQDN") that points to your jail, and the jail is reachable from the public Internet with ports 80 and 443 open to the jail--that is, if an Internet user browses to http://$FQDN, they'll reach your jail. -or-
2. Your DNS host has an API that's supported by acme.sh, allowing automated updates to your DNS records. Cloudflare has such an API, and provides DNS hosting for free. Many other DNS hosts' APIs are also supported; see https://github.com/Neilpang/acme.sh/tree/master/dnsapi

You'll specify which of these is the case in the script's configuration file. In addition, $FQDN needs to should resolve to your jail from inside your network; this is something you'd set up in your router configuration (in pfSense, for example, you'd set it up as a host override under Services -> DNS Resolver). Edit: It's no longer essential that $FQDN resolve to your jail from inside your network, but it's still preferred. You can use the IP address of your jail inside your network, but you'll get certificate errors that way.

This script installs and configures Nextcloud and all its dependencies.

Script and more details here: https://github.com/danb35/freenas-iocage-nextcloud

Edit: This has been tested on just one system right now, using the DNS validation for the cert. But it ran from start to finish, created the jail, installed everything, configured everything, obtained the cert, made the database changes, etc. Do not try this in production at this point.

Absolutely awesome

Sent from my HUAWEI NXT-AL10 using Tapatalk

 

[danb35]

Absolutely awesome

Thanks, but it was mostly a matter of automating the work you'd already done.

 

[blue_ice]

Thanks for the script but it's not working on FreeNAS-11.1-U4

JAIL_IP="192.168.1.102"
DEFAULT_GW_IP="192.168.1.1"
INTERFACE="re0"
POOL_PATH="/mnt/Volumen1"
JAIL_NAME="nextcloud"
TIME_ZONE="America/Toronto" # See http://php.net/manual/en/timezones.php
HOST_NAME="mynextcloud.ddns.net"
STANDALONE_CERT=1
DNS_CERT=0
TEST_CERT="--test"


re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=82099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether d0:50:99:98:3b:ce
hwaddr d0:50:99:98:3b:ce
inet 192.168.1.100 netmask 0xffffff00 broadcast 192.168.1.255


nextcloud successfully created!
Traceback (most recent call last):
File "/usr/local/bin/iocage", line 10, in <module>
sys.exit(cli())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/iocage/cli/create.py", line 145, in cli
basejail=basejail, empty=empty)
File "/usr/local/lib/python3.6/site-packages/iocage/lib/iocage.py", line 644, in create
exit_on_error=self.exit_on_error).create_jail()
File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_create.py", line 86, in create_jail
return self._create_jail(jail_uuid, location)
File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_create.py", line 445, in _create_jail
self.create_install_packages(jail_uuid, location, config)
File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_create.py", line 488, in create_install_packages
silent=True)
File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_start.py", line 64, in __init__
self.__start_jail__()
File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_start.py", line 127, in __start_jail__
vnet_interfaces = self.conf["vnet_interfaces"]
KeyError: 'vnet_interfaces'
nextcloud is not running, starting jail

 

[TrashKing]

FreeNas is configured to have jail root to:

Code:

/mnt/zvol/jail

Does POOL_PATH in nextcloud-config need to be inside the FreeNas jail root?

 

[TrashKing]

Got the same results as blueice on my VM running FreeNAS-11.1-U4

Code:

KeyError: 'vnet_interfaces'
nextcloud is not running, starting jail

 

[danb35]

KeyError: 'vnet_interfaces'
nextcloud is not running, starting jail

Bother. I'd seen that there were networking issues in 11.1-U3; apparently they didn't fix those in 11.1-U4. I'll see if I can figure out a workaround. Edit: Bug submitted as #30672, but I've attached a debug file so it will be marked private for the time being.

Does POOL_PATH in nextcloud-config need to be inside the FreeNas jail root?

No; in fact, it shouldn't be. It should be the path to the pool on which you want your data stored.

 

[TrashKing]

Bother. I'd seen that there were networking issues in 11.1-U3; apparently they didn't fix those in 11.1-U4. I'll see if I can figure out a workaround. Edit: Bug submitted as #30672, but I've attached a debug file so it will be marked private for the time being.

No; in fact, it shouldn't be. It should be the path to the pool on which you want your data stored.

Look forward to a working copy of this... Thanks for your effort

 

[TimvH]

To fix the iocage issues, you can manually update the script files using git.
cd /tmp
git clone --recursive https://github.com/iocage/iocage
cp -R iocage/iocage/lib/ /usr/local/lib/python3.6/site-packages/iocage/lib

Everything should be working now.

Edit:
Somehow there's something wrong with the database...
I also tried it with mariadb102-server but that does not change anything.
Complete install log with a little snip of the nextcloud folder structure(which is huge!) https://pastebin.com/eJVjK3ay
Hope anyone can find the problem here... been looking for hours.

 

[TrashKing]

To fix the iocage issues, you can manually update the script files using git.
cd /tmp
git clone --recursive https://github.com/iocage/iocage
cp -R iocage/iocage/lib/ /usr/local/lib/python3.6/site-packages/iocage/lib

Everything should be working now.

Edit:
Somehow there's something wrong with the database...
I also tried it with mariadb102-server but that does not change anything.
Complete install log with a little snip of the nextcloud folder structure(which is huge!) https://pastebin.com/eJVjK3ay
Hope anyone can find the problem here... been looking for hours.

Nextcloud looks promising but at this point installation of it is ridiculous.

 

[Yaguznal]

Hello everyone! Thank you danb35 for doing this. I've been trying to set up a nextcloud service from guides all over but always stranded on some error beyond my level of knowledge or googlefu.

It seems I am stranded beyond that line again. I have dug myself into apache and openssl for 2 days, reinstalled everything at least 20 times and have come to the point that I realize I need a hand from the pros so many thanks in advance! Also I am scared to make the browsergods angry and invite hackers because I disabled some essential openssl option.

Everything seems to install ok with no significant errors (png package timed out).

When finished, firefox barfs me a "SSL received a record that exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG" and opera "This site can’t provide a secure connection. www.homecommunity.linkpc.net sent an invalid response.".
Going to the HTTP version I get "It works!" So I'm guessing the includes virtualhost redirect does not work as intended.

I read somewhere that the error code means that there's a http stream coming trough the https channel.

I am able and willing to post logs though for some I will need a path or command.

I'm working on Kubuntu 17.10 and my FreeNAS-11.1-U4 box is in an other timezone (timezone of the box in config). Nuked my (as of yet empty) iocage and moved it to my SSD while I was at it. Script commit 7932cb6.
I read there was a networking bug in the iocage implementation of U3-U4 but so far that does not seem to be the case for me.

An other thing I noticed is that, for some reason, the promised password file in the root folder is missing, for me at least.

As a suggestion only it'd be sweet if you'd make us able to store the files, sql files and the port files in different datasets. If I'm not mistaking, sql benefits from the speed of my ssd but the files them self would fill it quickly so I'd store those on my raid array.
If I read other posts correctly the port dataset could be shared between iocages. Is that correct?

Never stop learning!

 

[danb35]

cd /tmp
git clone --recursive https://github.com/iocage/iocage
cp -R iocage/iocage/lib/ /usr/local/lib/python3.6/site-packages/iocage/lib

This seems to do the trick--I'll probably want to test a little more before I merge your pull request, but certainly the jail is created and has network access.

Somehow there's something wrong with the database.

I can't duplicate this with MariaDB101, which is what my script uses. From your log output, it appears that 10.2 wants the system .my.cnf file to be in /usr/local/etc/ or /usr/local/etc/mysql/, rather than in /var/db/mysql/ where my script puts it (and where 10.1 will look for it). You can make those changes on lines 127 and 130 of my script. I'll see if those locations work with 10.1 as well; if so I'll update the script accordingly. Edit: They don't. What happens if you try it with mariadb101 rather than 102?

 

[danb35]

SSL received a record that exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG

Interesting. It wasn't doing this previously, but I can now duplicate this behavior. It also doesn't seem to be handling php files properly. Time for some more digging...

 

[Yaguznal]

It also doesn't seem to be handling php files properly.

At one point it served me php files as a download, yes. I can't say if it's something I did and can't say what I did exactly.

 

[danb35]

At one point it served me php files as a download, yes.

After I made my changes to use PHP-FPM, I commented out the line to copy the mod_php config file. Apparently that (somehow) broke both PHP and SSL. If you uncomment line 124 (which is copying 001_mod_php.conf), it should work. Edit: Updated the script on GitHub to incorporate this change.

 

[TimvH]

This seems to do the trick--I'll probably want to test a little more before I merge your pull request, but certainly the jail is created and has network access.

I can't duplicate this with MariaDB101, which is what my script uses. From your log output, it appears that 10.2 wants the system .my.cnf file to be in /usr/local/etc/ or /usr/local/etc/mysql/, rather than in /var/db/mysql/ where my script puts it (and where 10.1 will look for it). You can make those changes on lines 127 and 130 of my script. I'll see if those locations work with 10.1 as well; if so I'll update the script accordingly. Edit: They don't. What happens if you try it with mariadb101 rather than 102?

The log output of mariadb101 was exactly the same, accidentally uploaded the mariadb102 log.
However with the latest commits the mysql error has changed. ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2 "No such file or directory")
I'll probably get this working soon and push a commit to my fork.
Edit: Somehow mariadb didn't get installed this time, time to clear my package cache and retry.

 

[danb35]

As a suggestion only it'd be sweet if you'd make us able to store the files, sql files and the port files in different datasets.

They're already in different datasets, but I think you meant on different pools. That should be do-able. Might take a couple of days before I get the change made, though.

If I read other posts correctly the port dataset could be shared between iocages. Is that correct?

Yes, that's correct--this is following @kjake's idea in the scripts he's done. Doesn't seem to be any sense in keeping multiple copies of that information.

 

[TimvH]

Somehow the configs didn't get mounted due to dirname giving ./config as output instead of the full path.
Changed line 25 of the script to SCRIPTPATH="$( cd "$(dirname "$0")" ; pwd -P )" and now it mounts the config dir fine.
Still getting the It works! page of apache but at least I'm coming closer.
Edit: forgot to merge upstream, testing again.

 

[cunningorb]

So which version should I be using of freenas now to test this script? U2 doesn't work with the repo as of 11pm eastern on 4/1. I tested on a fresh install.
I then updated to U4 so I will test that tonight if possible.

 

[danb35]

U2 doesn't work with the repo

?

I then updated to U4

That should work, with the workaround now suggested in the README.md file.