Blog Nextcloud and TrueNAS Collaborate to Help You Build Your Private Cloud

JoshDW19

Community Hall of Fame
Joined
May 16, 2016
Messages
1,077
Today we are announcing a partnership with Nextcloud to provide an officially supported integration with TrueNAS. Nextcloud and TrueNAS are the #1 Open Source platforms for team collaboration and software-defined storage, respectively. The Nextcloud software suite will plug into TrueNAS and both companies will offer support for the powerful combination.

image1-2-1024x560.png


Together, Nextcloud and TrueNAS combine to provide a very complete private cloud infrastructure with both data storage and a suite of team collaboration services like document creation, chat, email, conferencing, calendaring, and several others. The combination is Open Source and self-hosted for maximum privacy and security. Unlike public cloud services (e.g. G-Suite, Office 365), an organization’s data can be securely managed onsite without any third-party backdoors. The HA and integrated replication capabilities of TrueNAS allow very reliable infrastructure to be built that is less dependent on Internet access bandwidth or reliability.

Nextcloud Hub is a collaboration platform designed to be self-hosted for complete privacy and cost control. The applications included are:

Nextcloud Files: Share and sync documents, spreadsheets, presentations, photos, and any other type of documents. With Collabora Online (included), multiple users can edit documents in real-time. Data can be accessed via the web or Windows, Mac, Linux, iOS, and Android clients.​

Nextcloud Talk: Video and audio conferencing, combined with chat and whiteboarding, increase remote productivity in the new telecommuting era.​

Nextcloud Groupware: Calendars, webmail, and task management are integrated with Files and Talk so teams can collaborate both within and across organizations.​

TrueNAS is a software-defined storage platform which provides file, block, object, and app storage built on top of OpenZFS. The powerful enterprise-grade capabilities of TrueNAS include:

Data Management: Built into TrueNAS CORE, OpenZFS provides continuous integrity checks and self-healing, along with RAID functions, snapshots, clones, and replication of data.​

Integrated Security: Encryption of data-at-rest is managed with admin-provided keys or integration with enterprise KMIP servers. Integrated VPNs and encrypted replication provide protection from hackers.​

High Availability (HA): Downtime impacts productivity and is unacceptable to larger organizations. TrueNAS Enterprise provides dual-controller options (X-Series and M-Series) to deliver “five nines” availability (equivalent to downtime of less than 5 minutes per year).​

Scalability: Scale up to 20 PB in a single one-rack system or scale out to even larger systems with TrueNAS SCALE. Most importantly, there is no need to pay excessively for users with high capacity needs due to photos or videos.​

Unified Storage: While Nextcloud will manage a lot of data for the organization, the same TrueNAS may also manage NFS, SMB, iSCSI, or S3 data for other applications and backup systems.​



An official Nextcloud plugin for TrueNAS simplifies the installation and operation of Nextcloud. The plugin can be installed with a few clicks on a webUI to create a dataset and initiate the Nextcloud instance. The engineering teams of both companies will collaborate to ensure reliable operation and resolve any integration issues found. Users will have access to both the large Nextcloud and TrueNAS communities for feedback and questions.
image2-1-1024x419.png


The Nextcloud Plugin with TrueNAS Web UI​



The Nextcloud plugin is free and directly available for download within TrueNAS. Small businesses and extended families can set up their own private clouds in just a few clicks. For larger schools and organizations, an Enterprise support option is available starting at $8/month per user with no limits on the storage capacity or compute power per user. With the use of TrueNAS HA systems, these organizations will be able to build high-reliability solutions. TrueCommand can be used to manage distributed infrastructure deployments.

The initial Nextcloud plugin will be based on Nextcloud 22 and TrueNAS CORE 12.0-U6. Collabora will run as a Linux server, VM, or Kubernetes pod. Future versions of the plugin will feature integration with Collabora Online and integration with TrueNAS SCALE for scale-out operation. We look forward to working with both Nextcloud and TrueNAS communities to deliver a first-class Open Source experience.

Later this week, on Wednesday the 13th of October (8am Pacific = 5pm Central European Time), we will host a live Q&A session with Morgan Littlewood, Senior VP at iXsystems, and Jos Poortvliet, Marketing Director and Co-Founder of Nextcloud. It will stream live from the TrueNAS YouTube channel and everyone will be able to ask them anything, so stay tuned!

If you have any additional questions or need advice on a new TrueNAS or Nextcloud project, please contact us. We are standing by to help.

The post Nextcloud and TrueNAS Collaborate to Help You Build Your Private Cloud appeared first on iXsystems, Inc. - Enterprise Storage & Servers.

Continue reading...
 
Joined
Jan 4, 2014
Messages
1,644
Later this week, on Wednesday the 13th of October (8am Pacific = 5pm Central European Time), we will host a live Q&A session
Aussie users 11pm AWST, 2am AEDT
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
Does this mean that the Nextcloud plugin is going to start to, well, not suck?
 
Joined
Jan 4, 2014
Messages
1,644
Just wait until you see the updated plugin. ;)
This is only one half of the issue. Getting support for plugins is the flipside. That's the beaut thing about @danb35's scripted resource. I was always confident of being effectively supported in the discussion area for the resource. While there's an Applications and Plugins section of the forum, arms-length access to plugin developers has led to poor experiences with plugin support. It might also be worth considering configuring JIRA for bug reporting of iXsystems maintained plugins,
 
Last edited:

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,691
Agreed that bugs with plugin should be tracked on Jira.
Nextcloud engineers are collaborating so we have the expertise to resolve more complex issues.
 

stand

Dabbler
Joined
Sep 24, 2021
Messages
21
Is it too much to hope that this will make the Nextcloud team finally finish the 2—way sync implementation for mobile devices and on-demand sync for desktop? I mean it has only taken 5 years so far.
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,691
Is it too much to hope that this will make the Nextcloud team finally finish the 2—way sync implementation for mobile devices and on-demand sync for desktop? I mean it has only taken 5 years so far.
Is this a problem with Nextcloud on any platform or specific to TrueNAS? If it's the latter, then we can help. If it's the former, then its best to use the Nextcloud forum. This will be the general approach. It's best not to ask Nextcloud to fix bugs on TrueNAS either.

If its a joke.. apologies for taking it seriously :smile:
 

stand

Dabbler
Joined
Sep 24, 2021
Messages
21
Is this a problem with Nextcloud on any platform or specific to TrueNAS? If it's the latter, then we can help. If it's the former, then its best to use the Nextcloud forum. This will be the general approach. It's best not to ask Nextcloud to fix bugs on TrueNAS either.

If its a joke.. apologies for taking it seriously :smile:
Haha, it was sarcasm. But yes it is on the Nextcloud side. I am just frustrated that nothing gets done about it, since it is the only thing that keeps me from switching to Nextcloud.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
Nextcloud do seem to have a bad habit of being more focused on rolling out new features, than having those features actually work. E2EE is probably the most notorious example, but on-demand sync is another big one. And perhaps I'm just cynical, but I don't really anticipate that changing as a result of this partnership. But I'd settle for a stable, well-designed (minimally, that it allows you to store data (including the databases) in a defined location, and that it implements TLS sensibly), well-maintained (i.e., bugs are addressed promptly, it's kept up-to-date, and there's a defined update mechanism for users) plugin. Bonus points if it uses Caddy, but I doubt it will. Bonus points also if it handles being behind a reverse proxy without too much pain.
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,691
Thanks @danb35 appreciate the constructive criticism...if you get a chance to review, let us know where there are improvements to be made.
Can caddy be used as a front-end (reverse proxy) to the plugin?
 
Joined
Jan 4, 2014
Messages
1,644
Is it possible to turn this thread into a resource so it appears in the resources section of the forum? This will help consolidate all things to do with the plugin, now and in the future e.g. updates, discussions, reviews, etc.
 
Last edited:

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
Can caddy be used as a front-end (reverse proxy) to the plugin?
Caddy works well as a reverse proxy, and it's generally easy to configure (see https://github.com/danb35/freenas-iocage-caddy for some examples). It also handles TLS certs and termination automatically. Nextcloud would need a couple of settings made, though, and as I haven't used it behind a reverse proxy, I'm not too familiar with them--I think @Basil Hendroff could better address those.

As to potential improvements, compared to the current plugin, they're pretty much what I mentioned in my last post:
  • Let the user choose where data (including databases) will be stored
    • This would ideally be someplace outside of the jail, so that the user can relatively easily back it up as desired/required. A nice touch might be a nightly cron job to run mysqldump so there's a safe static file to back up.
  • Implement TLS in the jail
    • Even using a self-signed cert would put the relevant directives in the webserver config files, and the user could relatively easily edit them to point to a different cert. Having the jail HTTP-only means the user has to implement HTTPS from scratch, and there are lots of threads here (from users who didn't know to use my script) about having problems with that.
  • It needs to be kept up-to-date
    • Nextcloud has lots of moving parts--a webserver, a database server, and PHP as the major ones, but there are bunch of smaller things. All need to be reasonably up-to-date for security reasons, as does Nextcloud itself. Nextcloud has its own updater built-in, and if the plugin is built in such a way that it won't break things, that's good--but all the other software needs to be fairly current as well.
  • Bugs need to be addressed promptly
    • It hasn't been my impression that there's been a very high priority placed on handling bugs with plugins. Maybe the difference between "iXSystems" and "community" plugins is part of this, but I remember opening a ticket, with a proposed patch and PR, against the Nextcloud plugin, and waiting over six months for any response at all.
  • Caddy
    • Yeah, I'm a Caddy fanboy. Why? Well, compare its config file for Nextcloud (all 41 lines of it) with what nginx or Apache require. And it handles TLS automagically. It isn't perfect; the biggest drawback I can think of is if you want to obtain your cert using DNS validation. In that case, apparently as a result of its being written in Go, you can't just download a plugin; you need to either build Caddy yourself (not too hard; it's what my script does), or download a compiled binary including that plugin.
  • Reverse proxy compatibility
    • This is likely tricky, and might conflict with implementing TLS as suggested above. But it would be good to have the plugin set up in such a way as to work behind a reverse proxy.
Doubtless there are others, but those are what come to mind.
 

Heracles

Wizard
Joined
Feb 2, 2018
Messages
1,401
About running Nextcloud behind a reverse proxy.... I do it here, using pfSense and HAProxy as my frontend. The Nextcloud instance is the official Docker container. It runs in a Linux VM in ESXi. Storage is mapped to that Linux over NFS and then re-mounted inside the container as a volume. That way, Nextcloud is unaware that these data are actually remote.

Things to mention are :
--Every time the container is updated, I need to re-configure its built-in Apache server to extract and use the X-Forwarded-For address in its logs
--In the config file, the required settings are "trusted_proxies" and "trusted_domains", as well as overwriteprotocol because HAProxy calls back to it over clear text. Because the config files are in dedicated docker volume, they survive container updates.
--My database is MarriaDB running in a separate container (and actually a separate host).
 

morganL

Captain Morgan
Administrator
Moderator
iXsystems
Joined
Mar 10, 2018
Messages
2,691
Caddy works well as a reverse proxy, and it's generally easy to configure (see https://github.com/danb35/freenas-iocage-caddy for some examples).
  • Caddy
    • Yeah, I'm a Caddy fanboy. Why? Well, compare its config file for Nextcloud (all 41 lines of it) with what nginx or Apache require. And it handles TLS automagically. It isn't perfect; the biggest drawback I can think of is if you want to obtain your cert using DNS validation. In that case, apparently as a result of its being written in Go, you can't just download a plugin; you need to either build Caddy yourself (not too hard; it's what my script does), or download a compiled binary including that plugin.
  • Reverse proxy compatibility
    • This is likely tricky, and might conflict with implementing TLS as suggested above. But it would be good to have the plugin set up in such a way as to work behind a reverse proxy.

It seems like the middle ground is to make sure that the Nextcloud plugin works well behind Caddy as a reverse proxy??
Caddy provides all the TLS capabilities. Is this viable?
 
Joined
Jan 4, 2014
Messages
1,644
It seems like the middle ground is to make sure that the Nextcloud plugin works well behind Caddy as a reverse proxy??
Caddy provides all the TLS capabilities. Is this viable?
I believe this is the way to go. Apply the KISS principle. Allow Caddy to handle TLS termination and automatic certificate renewal. A mechanism to encrypt the communications path between nginx in the NC plugin and the Caddy server can be described later. Having nginx handle TLS termination initially complicates troubleshooting.

Caddy works well as a reverse proxy, and it's generally easy to configure (see https://github.com/danb35/freenas-iocage-caddy for some examples). It also handles TLS certs and termination automatically. Nextcloud would need a couple of settings made, though, and as I haven't used it behind a reverse proxy, I'm not too familiar with them

Details for the settings and for placing @danb35's NC resource behind an RP can be found in steps 1 and 2 of the resource Nextcloud and OnlyOffice Integration. More rigourous feedback on setting up the NC plugin behind an RP to follow.
 
Last edited:
Joined
Jan 4, 2014
Messages
1,644
Several scenarios were considered in testing the NC plugin behind an RP:
  1. NC plugin on CORE using NAT
  2. NC plugin on CORE using DHCP
  3. NC app on SCALE
Scenario 1: NC plugin using NAT

This is the default network setting for the NC plugin jail.

tn58.jpg

The plugin installs successfully with the following install notes:

Code:
Install Notes:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
No certificates found.
Saving debug log to /var/log/letsencrypt/letsencrypt.log
No certificates found.
Generating a RSA private key
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
.................................................................................................................................................................+++++
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
.............+++++
No certificates found.
writing new private key to '/usr/local/etc/letsencrypt/live/truenas/root.key'
-----
No certificates found.
Generating a RSA private key
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
......................................................+++++
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
No certificates found.
.......................................................................+++++
writing new private key to '/usr/local/etc/letsencrypt/live/truenas/server.key'
No certificates found.
-----
No certificates found.
Signature ok
subject=O = TrueNAS (Nextcloud), CN = localhost
No certificates found.
Getting CA Private Key
You can install the following CA on your devices to trust the TLS certificate: /usr/local/etc/letsencrypt/live/truenas/root.cer
Getting CA Private Key
nginx_enable: -> YES
Getting CA Private Key
mysql_enable: -> YES
Getting CA Private Key
php_fpm_enable: -> YES
Getting CA Private Key
redis_enable: -> YES
Getting CA Private Key
fail2ban_enable: -> YES
Getting CA Private Key
Performing sanity check on nginx configuration:
Getting CA Private Key
Starting nginx.
Getting CA Private Key
Performing sanity check on php-fpm configuration:
Getting CA Private Key
Starting php_fpm.
Getting CA Private Key
Starting mysql.
Getting CA Private Key
Starting redis.
Getting CA Private Key
Starting redis.
mysqladmin: [Warning] Using a password on the command line interface can be insecure.
Warning: Since password will be sent to server in plain text, use ssl connection to ensure password safety.
Starting redis.
mysql: [Warning] Using a password on the command line interface can be insecure.
Nextcloud was successfully installed
mysql: [Warning] Using a password on the command line interface can be insecure.
Set mode for background jobs to 'cron'
mysql: [Warning] Using a password on the command line interface can be insecure.
System config value trusted_domains => 1 set to string 10.1.1.14
mysql: [Warning] Using a password on the command line interface can be insecure.
contacts 4.0.3 installed
mysql: [Warning] Using a password on the command line interface can be insecure.
contacts enabled
mysql: [Warning] Using a password on the command line interface can be insecure.
calendar 2.3.4 installed
mysql: [Warning] Using a password on the command line interface can be insecure.
calendar enabled
mysql: [Warning] Using a password on the command line interface can be insecure.
notes 4.1.1 installed
mysql: [Warning] Using a password on the command line interface can be insecure.
notes enabled
mysql: [Warning] Using a password on the command line interface can be insecure.
deck 1.5.3 installed
mysql: [Warning] Using a password on the command line interface can be insecure.
deck enabled
mysql: [Warning] Using a password on the command line interface can be insecure.
spreed 12.1.2 installed
mysql: [Warning] Using a password on the command line interface can be insecure.
spreed enabled
mysql: [Warning] Using a password on the command line interface can be insecure.
mail 1.10.5 installed
mysql: [Warning] Using a password on the command line interface can be insecure.
mail enabled
mysql: [Warning] Using a password on the command line interface can be insecure.
Server ready
mysql: [Warning] Using a password on the command line interface can be insecure.
Admin Portal:
http://10.1.1.14:8282

Accessing the jail at http://10.1.1.14:8282 appears to cause a rewrite to port 8283.

tn60.jpg

Comparing the plugin config.php with the config.php for a working NC instance (using Dan's script) behind an RP, I note there are crucial parameters missing. I inject these into the plugin config.php.

Code:
root@ncnat:~ # su -m www -c 'php /usr/local/www/nextcloud/occ config:system:set overwriteprotocol --value="https"'
System config value overwriteprotocol set to string https
root@ncnat:~ # su -m www -c 'php /usr/local/www/nextcloud/occ config:system:set overwritehost --value="ncnat.udance.com.au"'
System config value overwritehost set to string ncnat.udance.com.au
root@ncnat:~ # su -m www -c 'php /usr/local/www/nextcloud/occ config:system:set trusted_domains 2 --value="ncnat.udance.com.au"'
System config value trusted_domains => 2 set to string ncnat.udance.com.au
root@ncnat:~ # su -m www -c 'php /usr/local/www/nextcloud/occ config:system:set overwrite.cli.url --value="http://ncnat.udance.com.au/"'
System config value overwrite.cli.url set to string http://ncnat.udance.com.au/


In the Caddyfile for the Caddy RP, I map ncnat.udance.com.au to 10.1.1.14:8282. Attempting to connect to the NC instance, I'm greeted with the following:

tn61.jpg

The appending of the port to the subdomain is unexpected. I suspect this is something the nginx webserver is injecting in.

Altering the Caddy code to map to port 8283 instead results in the following:

tn62.jpg

Scenario 1: Test result: FAIL

Scenario 2 and 3 testing to follow.
 
Last edited:

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,737
@Basil Hendroff try setting port_in_redirect off; in the plugin's nginx.conf.
 
Joined
Jan 4, 2014
Messages
1,644
@Basil Hendroff try setting port_in_redirect off; in the plugin's nginx.conf.

Good call. That stopped the port redirect, but led to a 502 error. I won't be able to dive into this further now as I have to head out in a couple of hours. Before then, I'll try to get scenario 2 and 3 test results out. The merits of these can be discussed with the community later.

tn64.jpg
 
Top