Scripted installation of Nextcloud 28 in iocage jail 2018-03-23

[danb35]

WOW seems like a lot of work maybe I'm missing something.

Yeah, maybe you are. Compare your process to:

• Download script
• Write config file of a half-dozen lines
• Run script

Edit: and not only does the script create the jail, install and configure all the software, it also gets a cert from Let's Encrypt and sets that up to automatically renew.

 

[glauco]

Thank you danb35 for developing this script... it makes the difference between giving Nextcloud a try and giving up.
My knowledge of web servers in general is very limited so please, be patient with me!
This is the content of my nextcloud-config file:

Code:

JAIL_IP="10.0.0.14"
DEFAULT_GW_IP="10.0.0.1"
INTERFACE="lagg0"
POOL_PATH="/mnt/MyVolume"
JAIL_NAME="nextcloud"
TIME_ZONE="Europe/Rome" # See http://php.net/manual/en/timezones.php
HOST_NAME="nc.soon.it"
STANDALONE_CERT=1
DNS_CERT=0
TEST_CERT="--test"

I noticed that the IP address I set in the config file (10.0.0.14) was added to the network interface lagg0. Is that how it's supposed to be?
lagg0 is a link aggregation of igb0 and igb1, could that cause issues?
This is my ifconfig output:

Code:

igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>
		ether 0c:c4:7a:e2:05:12
		hwaddr 0c:c4:7a:e2:05:12
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect (1000baseT <full-duplex>)
		status: active
igb1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>
		ether 0c:c4:7a:e2:05:12
		hwaddr 0c:c4:7a:e2:05:13
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect (1000baseT <full-duplex>)
		status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
		options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
		inet6 ::1 prefixlen 128
		inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
		inet 127.0.0.1 netmask 0xff000000
		nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
		groups: lo
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>
		ether 0c:c4:7a:e2:05:12
		inet 10.0.0.11 netmask 0xffffff00 broadcast 10.0.0.255
		inet 10.0.0.14 netmask 0xffffff00 broadcast 10.0.0.255
		nd6 options=9<PERFORMNUD,IFDISABLED>
		media: Ethernet autoselect
		status: active
		groups: lagg
		laggproto lacp lagghash l2,l3,l4
		laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
		laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
		ether 02:c0:ec:4f:a7:00
		nd6 options=1<PERFORMNUD>
		groups: bridge
		id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
		maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
		root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
		member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 8 priority 128 path cost 2000
		member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 7 priority 128 path cost 2000
		member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 6 priority 128 path cost 2000
		member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
				ifmaxaddr 0 port 4 priority 128 path cost 10000
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ee:50:00:06:0a
		hwaddr 02:ee:50:00:06:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
		groups: epair
epair1a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ee:50:00:07:0a
		hwaddr 02:ee:50:00:07:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
		groups: epair
epair2a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
		options=8<VLAN_MTU>
		ether 02:ee:50:00:08:0a
		hwaddr 02:ee:50:00:08:0a
		nd6 options=1<PERFORMNUD>
		media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
		status: active
		groups: epair

And these are the last lines of the output of your script.
I'm pretty sure I've configured port-forwarding correctly on my pfSense router.
Yet, https://www.grc.com/x/ne.dll?rh1dkyd2 reports that port 80 is not open. Perhaps because the web server hasn't been deployed?
What am I doing wrong?

Code:

[Sun May 27 18:22:31 CEST 2018] Using stage ACME_DIRECTORY: https://acme-staging.api.letsencrypt.org/directory
[Sun May 27 18:22:31 CEST 2018] Standalone mode.
netstat: kvm not available: /dev/mem: No such file or directory
[Sun May 27 18:22:32 CEST 2018] Registering account
[Sun May 27 18:22:33 CEST 2018] Registered
[Sun May 27 18:22:33 CEST 2018] ACCOUNT_THUMBPRINT='167M4d3F8em-znrB1qcpxaK8Kuk_IlKhzOT4yENEbx8'
[Sun May 27 18:22:33 CEST 2018] Creating domain key
[Sun May 27 18:22:33 CEST 2018] The domain key is here: /root/.acme.sh/nc.soon.it/nc.soon.it.key
[Sun May 27 18:22:33 CEST 2018] Single domain='nc.soon.it'
[Sun May 27 18:22:33 CEST 2018] Getting domain auth token for each domain
[Sun May 27 18:22:33 CEST 2018] Getting webroot for domain='nc.soon.it'
[Sun May 27 18:22:33 CEST 2018] Getting new-authz for domain='nc.soon.it'
[Sun May 27 18:22:34 CEST 2018] The new-authz request is ok.
[Sun May 27 18:22:34 CEST 2018] Verifying:nc.soon.it
[Sun May 27 18:22:34 CEST 2018] Standalone mode server
[Sun May 27 18:22:39 CEST 2018] Pending
[Sun May 27 18:22:41 CEST 2018] Pending
[Sun May 27 18:22:43 CEST 2018] Pending
[Sun May 27 18:22:46 CEST 2018] Pending
[Sun May 27 18:22:48 CEST 2018] nc.soon.it:Verify error:Fetching http://nc.soon.it/.well-known/acme-challenge/FKACG8-vI4u70BAFlN7GSyVkT8sxR2aiVMkyvz5AOXk: Timeout during connect (likely firewall problem)
[Sun May 27 18:22:48 CEST 2018] Please add '--debug' or '--log' to check more details.
[Sun May 27 18:22:48 CEST 2018] See: https://github.com/Neilpang/acme.sh/wiki/How-to-debug-acme.sh
True












* Stopping nextcloud
  + Running prestop OK
  + Stopping services OK
  + Removing jail process OK
  + Running poststop OK
* Starting nextcloud
  + Started OK
  + Starting services OK










[Sun May 27 18:23:30 CEST 2018] Using stage ACME_DIRECTORY: https://acme-staging.api.letsencrypt.org/directory
[Sun May 27 18:23:31 CEST 2018] Single domain='nc.soon.it'
[Sun May 27 18:23:31 CEST 2018] Getting domain auth token for each domain
[Sun May 27 18:23:31 CEST 2018] Getting webroot for domain='nc.soon.it'
[Sun May 27 18:23:31 CEST 2018] Getting new-authz for domain='nc.soon.it'
[Sun May 27 18:23:32 CEST 2018] The new-authz request is ok.
[Sun May 27 18:23:32 CEST 2018] Verifying:nc.soon.it
[Sun May 27 18:23:35 CEST 2018] Pending
[Sun May 27 18:23:37 CEST 2018] Pending
[Sun May 27 18:23:40 CEST 2018] Pending
[Sun May 27 18:23:42 CEST 2018] Pending
[Sun May 27 18:23:44 CEST 2018] nc.soon.it:Verify error:Fetching http://nc.soon.it/.well-known/acme-challenge/ylJ3kG4UsGdVXP_WjfbPSYKdc2_Did77QKmGd6VdIyU: Timeout during connect (likely firewall problem)
[Sun May 27 18:23:44 CEST 2018] Please add '--debug' or '--log' to check more details.
[Sun May 27 18:23:44 CEST 2018] See: https://github.com/Neilpang/acme.sh/wiki/How-to-debug-acme.sh
True


Nextcloud is not installed - only a limited number of commands are available
Nextcloud was successfully installed

System config value logtimezone set to string Europe/Rome

System config value log_type set to string file

System config value logfile set to string /var/log/nextcloud.log

System config value loglevel set to string 2

System config value logrotate_size set to string 104847600

System config value memcache.local set to string \OC\Memcache\APCu

System config value redis => host set to string /tmp/redis.sock

System config value redis => port set to integer 0

System config value memcache.locking set to string \OC\Memcache\Redis

System config value htaccess.RewriteBase set to string /

.htaccess has been updated

System config value trusted_domains => 1 set to string nc.soon.it

System config value trusted_domains => 2 set to string 10.0.0.14

encryption enabled

Encryption enabled

Default module: OC_DEFAULT_MODULE

Encryption disabled

Set mode for background jobs to 'cron'


Successfully removed mount from nextcloud's fstab
Installation complete!
Using your web browser, go to https://nc.soon.it to log in
Default user is admin, password is NeKcRNha3CfLhsGW

Database Information
--------------------
Database user = nextcloud
Database password = uwMpPhBJWxuYZARbhxC/jA==
The MariaDB root password is 8/pT5uK23ijv1ejxGDEQRQ==

All passwords are saved in /root/db_password.txt

Thank you so much!!

 

[danb35]

I noticed that the IP address I set in the config file (10.0.0.14) was added to the network interface lagg0. Is that how it's supposed to be?

I'd expect so, since you gave lagg0 as your interface.

Yet, https://www.grc.com/x/ne.dll?rh1dkyd2 reports that port 80 is not open. Perhaps because the web server hasn't been deployed?

Yes, that would be the (or at least "a") reason*. That's the reason that acme.sh runs in standalone mode the first time it tries to obtain a cert--it spins up its own mini-web server for validation purposes. @dureal99d, whose how-to I used as a starting point, has you spin up the web server, get the cert, then update the server's configuration to use that cert, which is significantly more complicated to script--but it would end up making troubleshooting a bit easier.

* Other reasons include incorrect port-forwarding configuration on your router, incorrect DNS configuration, your ISP blocking port 80, and grc.com being flaky.

The ultimate answer is probably going to be to blow away the jail the script created and start again, since the script doesn't have any logic in it to pick up at a certain point. But before you do that, we need to figure out why you're seeing the timeout in certificate validation. Try this: Go to https://www.sslchecker.com/csr/self_signed and generate a self-signed cert for your hostname (nc.soon.it). Place the certificate in your jail at /usr/local/etc/pki/tls/certs/fullchain.pem, and the private key at /usr/local/etc/pki/tls/private/privkey.pem. Then try to start apache ( service apache24 start). If that runs without errors, try a few tests:

• From inside your network, see if you can browse to your jail by its IP address
• From inside your network, see if you can browse to your jail by its hostname
• From outside your network, see if you can browse to your hail by its hostname
• Try the portscan from grc.com again

 

[glauco]

Wow, I'm amazed!!
I did what you told me and - just like magic - it works now!
All the tests you told me to take were passed except for the portscan from grc.com, so I tried other online port-scanners without success. Then I remembered... it must be because I have configured pfBlockerNG on my pfSense router to drop unsolicited inbound traffic to the WAN interface from all countries except mine!
I could turn off pfBlockerNG altogether while I relaunch your script but then the auto-renewal wouldn't work so, do you think that whitelisting the Letsencrypt.org IP address (23.12.99.165) would be enough?
Thank you so much!

 

[danb35]

do you think that whitelisting the Letsencrypt.org IP address (23.12.99.165) would be enough?

No. Let's Encrypt have repeatedly stated their intent to validate from a wide range of IP addresses and geographical locations, so at least at the time of cert issuance or renewal, the whole world needs to be able to connect on port 80. If that isn't acceptable, look into using DNS validation--many DNS hosts have supported APIs, and then you could set your firewall however you wanted.

 

[glauco]

Thank you! Now and forever!

 

[Yaguznal]

I was having the same trouble as glauco. Had the right port forwards in place yet apache wouldn't start because of missing certificates. Tried your trick, @danb35, and it came to life. Discovered that my isp blocks 80. 443 and 23 after that. I disabled that protection and yet, I can not reach my jail from the outside world, only from inside my lan. Do you think it needs some time before the change comes trough?
When it does, which command can I run to start the letsencrypt installation without rerunning the whole script? I found the piece of code in your script but unfortunately there's too much references I don't get yet.

You are awesome dan. Thanks.

 

[danb35]

Discovered that my isp blocks 80. 443 and 23 after that. I disabled that protection and yet, I can not reach my jail from the outside world, only from inside my lan. Do you think it needs some time before the change comes trough?

It's hard to say. In my own experience, ISPs who block port 80 won't generally unblock it just because the customer asks them to; they make you upgrade to a more-expensive account (which is why I have business-class Internet service at my home). In any event, port 23 is meaningless, and there's no reason you should want it open (other than that you don't want your ISP blocking anything).

But if there's nothing running in your jail on port 80, anything you try to reach port 80 from the outside isn't going to work. And since your attempt to create a certificate failed, Apache won't start, so there's nothing running on port 80. So I'll suggest the same thing to you that I did to @glauco--generate a self-signed cert, put the files where the Apache config expects to see them, and start Apache. Then see if it's reachable from the outside. If it is, you can proceed to get a cert from Let's Encrypt using this command from inside the jail:
acme.sh --issue -d ${HOST_NAME} -w /usr/local/www/apache24/data -k 4096 --fullchain-file /usr/local/etc/pki/tls/certs/fullchain.pem --key-file /usr/local/etc/pki/tls/private/privkey.pem --reloadcmd "service apache24 reload"
The only thing you need to change in that is ${HOST_NAME}; you'd replace that with your FQDN.

 

[sdgenxr]

It appears that I'm getting the same timeout message as @glauco. I can navigate into the jail and add the self-signed certs as per:

Go to https://www.sslchecker.com/csr/self_signed and generate a self-signed cert for your hostname (nc.soon.it). Place the certificate in your jail at /usr/local/etc/pki/tls/certs/fullchain.pem, and the private key at /usr/local/etc/pki/tls/private/privkey.pem.

However, when I try and restart apache I get:
root@server:/mnt/iocage/jails/nextcloud/root # service apache24 start
Cannot 'start' apache24. Set apache24_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'.

Yet apache24 is already enabled (rc.conf):
host_hostname="nextcloud"
cron_flags="$cron_flags -J 15"

# Disable Sendmail by default
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Run secure syslog
syslogd_flags="-c -ss"

# Enable IPv6
ipv6_activate_all_interfaces="YES"
apache24_enable="YES"
mysql_enable="YES"
redis_enable="YES"
php_fpm_enable="YES"

 

[danb35]

root@server:/mnt/iocage/jails/nextcloud/root # service apache24 start

You need to run this command using the jail's shell, not the FreeNAS shell. Do iocage console nextcloud, then try again.

 

[sdgenxr]

Wow, thanks for the quick reply!!!
Here's the error that I'm getting now:
[root@nextcloud /usr/local/etc/pki/tls/private]# service apache24 start
Performing sanity check on apache24 configuration:
Syntax OK
Starting apache24.
/usr/local/etc/rc.d/apache24: WARNING: failed to start apache24

By the way, thank you very much for all the support you are providing! I know that there can be a lot of extra work involved when putting together something like this and greatly appreciate your hard work!

 

[danb35]

/usr/local/etc/rc.d/apache24: WARNING: failed to start apache24

Strange that it didn't give any further indication of the error (edit: you might find more in /var/log/httpd-error.log). Did you put the cert files in the jail, or in the base FreeNAS system? They should be in the jail.

 

[sdgenxr]

Ok, quick recap of where I'm at now:

1. Wiped out jail with iocage destroy nextcloud
2. Removed iocage datasets installed from the previous script through the web gui
3. Removed other user-defined datasets from the previous installation through the web gui
4. Removed rm -r /tmp directory created to download git repository from the previous installation
5. Created new datasets that correspond to user-defined datasets from the nextcloud-config
6. Created mkdir /tmp and navigated to cd /tmp directory
7. Downloaded latest git repository git clone https://github.com/danb35/freenas-iocage-nextcloud
8. Navigated to cd freenas-iocage-nextcloud
9. Created nano nextcloud-config and provided appropriate info as per previous instructions
10. Executed ./nextcloud-jail.sh
11. Made breakfast while script did its amazing thing
12. Certificates timed out similar to @glauco previous post
13. Generated a self-signed cert from https://www.sslchecker.com/csr/self_signed
14. SSH'd into the nextcloud jail with iocage console nextcloud
15. Pasted the certificate to nano /usr/local/etc/pki/tls/certs/fullchain.pem
16. Pasted the private key to nano /usr/local/etc/pki/tls/private/privkey.pem
17. Started apache with service apache24 start
18. Apache successfully started
19. Ran port scan from http://www.whatsmyip.org/port-scanner/server/ with only port 443 open
20. Cannot browse to jail from local IP address inside my network
21. Cannot browse to jail from hostname inside my network
22. Cannot browse to jail from hostname outside my network
23. Cannot ping hostname
24. Can ping local jail IP

Reviewing /var/log/httpd-error.log after an Apache restart provides:
[Sat Jun 02 12:32:17.032022 2018] [mpm_event:notice] [pid 23626:tid 34397577216] AH00491: caught SIGTERM, $
[Sat Jun 02 12:32:17.304644 2018] [ssl:warn] [pid 23892:tid 34397577216] AH01873: Init: Session Cache is n$
[Sat Jun 02 12:32:17.308832 2018] [mpm_event:notice] [pid 23892:tid 34397577216] AH00489: Apache/2.4.33 (F$
[Sat Jun 02 12:32:17.308942 2018] [core:notice] [pid 23892:tid 34397577216] AH00094: Command line: '/usr/l$


Another roadblock to overcome. Any thoughts on where to proceed next?

 

[danb35]

Ran port scan from http://www.whatsmyip.org/port-scanner/server/ with only port 443 open

This is going to prevent you from obtaining a Let's Encrypt cert unless you use DNS validation--that process must have port 80 open to function.

• Cannot browse to jail from local IP address inside my network

• Cannot browse to jail from hostname inside my network

• Cannot browse to jail from hostname outside my network

What happens when you try in each case, and are you trying to connect using HTTPS?

 

[sdgenxr]

After some digging around, it appears that my ISP is blocking port 80 now. I say now because it was open the other week and I was able to test it by successfully forwarding my hostname to another jail on my server.

I can connect to the Nextcloud jail from within my network using HTTPS and the jail IP, and cannot seem to connect any other way.

 

[danb35]

After some digging around, it appears that my ISP is blocking port 80 now.

Unfortunately, that isn't unusual for residential ISPs. It will prevent you from getting a Let's Encrypt cert using HTTP validation--you'd need to use DNS validation instead, which means you need to use a compatible DNS host.

cannot seem to connect any other way.

Again, what happens when you try?

 

[jasemo]

Hey sdgenxr,

I had a bit of difficulty with my install as I'm stuck on with both a Dynamic IP AND a blocked port 80. The block means I had to use DNS validation but the Dynamic IP complicates that.

I spent a lot of time looking for a service that would integrate with acme.sh and also take DDNS updates, but couldn't find any free ones. Either they'd take DDNS updates from my DD-WRT router but NOT work with acme.sh or visa-versa. However, after a lot of help from danb35 I found the trick was to get Cloudflare account - which works well with acme.sh - and use DNS-O-Matic in the middle to take DNS updates from my router and send them on to Cloudflare.

So basically:
DD-WRT → DNS-O-Matic → Cloudflare

There were a couple of settings that needed to be checked in Cloudflare but my Nextcloud install has been rock steady for a couple of months now (touch wood). I'm not sure if you have the complication of a Dynamic IP but I know once I got on to the path with Cloudflare & acme.sh, I significantly lessened the amount of hair pulling out I was doing.

There are far more qualified people to help in this thread, but let me know if I can explain anything more about this convoluted solution.

:)

 

[sdgenxr]

Again, what happens when you try?

It times out.

@jasemo Thanks for the information. I'll definitely look into your solution when I can find more time to play around with my server.

Special thaks to @danb35 for all your help and putting together this script for the community!

 

[Yaguznal]

The ports deblocked after a modem restart and the installation is working at last. :D
Is there a way to read the log of the installation?
How do I change to a real certificate if the test certificate succeeded?

 

[danb35]

Is there a way to read the log of the installation?

The installation log isn't saved to disk anywhere, but you should be able to scroll back in your SSH client to see whatever you need.

How do I change to a real certificate if the test certificate succeeded?

Depends on how you got the cert in the first place, but the simple answer is that you do the same thing the script did to get the test cert, but without the --test flag. If you used HTTP validation, that would be:
/root/.acme.sh/acme.sh --issue --home "/root/.acme.sh" -d ${HOST_NAME} -w /usr/local/www/apache24/data -k 4096 --fullchain-file /usr/local/etc/pki/tls/certs/fullchain.pem --key-file /usr/local/etc/pki/tls/private/privkey.pem --reloadcmd "service apache24 reload"