SOLVED PfSense/TrueNAS/TrueCharts Nextcloud (or any SSL)

[Robert Thomspon]

OK, at my wits end here.
Ive got a PfSense box handling my incoming traffic. Ive tried to get it to forward traffic straight to a nextcloud instance (or any SSL traffic, its not specific to nextcloud). Ive used HAProxy and ive used just straight port forwarding, to no avail. Ive tried having all traffic sent through traefik as a pod (theres only a web interface to view traefik, but no way that ive found to actually modify any of the internal calls inside of traefik (like modifying a yaml or sending CLI commands) and because of the backend being locked down on traefik, it seems that you cant do any advanced routing or troubleshooting with traefik (it either does or does not work and there is no real way to figure out how or why)...

ANYWAY, i also use cloudflare to direct my domain and subdomains to my in home server.

Has ANYONE successfully gotten PfSense and TrueNAS Scale to work correctly with Scale's pods?

What i WANT is to go to cloud.mydomain.com and have it direct me to my home cloud instance at 192.168.2.2:9443 (or whatever port, i dont care)
I also dont particularly care about the traffic being encrypted once it hits my network (its preferable but way more a WANT than a NEED, but external comms still need to be encrypted). I have also tried official builds of nextcloud and the truecharts nextcloud (truecharts i have had almost no success with, official release, i can get the name to resolve to an internal IP but not load what its supposed to. (and yes, ive read through and followed truecharts supposed manual)

ANY help?

TrueNAS Scale 12.02RC2, applied truecharts hotfix, tried with and without treafik, tried with node, tried with cluster (node gets it looking like im resolving to the correct internal IP, but fails to connect). Ive tried with and without ingress, ive tried every combination i can think of on PfSense with and without HAProxy, ive tried different (less strict) settings in cloudflare... i feel like ive gotten REALLY close to gettting this to work... but just cant seem to make that last step

Oh, ive also tested through canyouseeme and tested my certs at SSL Labs (coming back with B grades because i had TLS 1.0 enabled)
Any help is greatly appreciated!!

 

[Robert Thomspon]

OK, so, i FINALLY figured it out.

Ill have a write up in a few days for everyone else that needs one (if youre hurting to know how to do it, PM me and ill reply as fast as i can)

 

[nubian122]

OK, so, i FINALLY figured it out.

Ill have a write up in a few days for everyone else that needs one (if youre hurting to know how to do it, PM me and ill reply as fast as i can)

I'm interested in the write up. Also you will get B grades on the SSL check because Cloudflare uses internal certs oddly enough and not LetsEncrypt ones for it.

 

[sstruke]

OK, so, i FINALLY figured it out.

Ill have a write up in a few days for everyone else that needs one (if youre hurting to know how to do it, PM me and ill reply as fast as i can)

Can you help me because I don't know how to configure this

 

[truecharts]

We generally highly advice keeping using the loadbalancer inside of Kubernetes by using Kubernetes Ingress with, for example, our Traefik App.
That should automatically configure everything as-needed.

 

[sstruke]

We generally highly advice keeping using the loadbalancer inside of Kubernetes by using Kubernetes Ingress with, for example, our Traefik App.
That should automatically configure everything as-needed.

Ok Thanks I'll try this

 

[sstruke]

We generally highly advice keeping using the loadbalancer inside of Kubernetes by using Kubernetes Ingress with, for example, our Traefik App.
That should automatically configure everything as-needed.

Works great, thanks for the advice

 

[Imslow]

OK, so, i FINALLY figured it out.

Ill have a write up in a few days for everyone else that needs one (if youre hurting to know how to do it, PM me and ill reply as fast as i can)

Dear Robert Thompson, I would be very happy to hear from your solution. I am running a similar setup as you do (HAProxy on pfsense+ 23.01) and can't figure out how to configure TrueCharts Nextcloud apps parameters to use my FQDN and pass A+ security score.

 

[truecharts]

Dear Robert Thompson, I would be very happy to hear from your solution. I am running a similar setup as you do (HAProxy on pfsense+ 23.01) and can't figure out how to configure TrueCharts Nextcloud apps parameters to use my FQDN and pass A+ security score.

Our Nextcloud App has an A+ SSL labs score out-of-the-box, when used with Traefik and Ingress.
(and usually when up-to-date also A+ from Nextcloud security scan)

Traefik and Ingress is 100% working with TrueCharts Nextcloud and actually the only supported way of it being setup.
We've staff available using HA proxy on discord as well, though officially we don't offer support for putting other reverse proxies in front of it.

In all cases, however, you should be running Traefik and Ingress regardless of weither you put another reverse proxy in front of our APps.

 

[Imslow]

Thanks for the prompt reply. Looks like I will need to learn about Traefik.

 

[truecharts]

Thanks for the prompt reply. Looks like I will need to learn about Traefik.

It's important to note that we only use traefik as an ingress provider.
Besides the settings layed out on our website, there is not (much) traefik specifik knowhow that is required or even usefull besides those instructions and middlewares if you want to use those.

In the future it might be best to reach out to our support staff directly on Discord, we're a seperate project and don't regularly check this forum or offer support here.

 

[Imslow]

It's important to note that we only use traefik as an ingress provider.
Besides the settings layed out on our website, there is not (much) traefik specifik knowhow that is required or even usefull besides those instructions and middlewares if you want to use those.

In the future it might be best to reach out to our support staff directly on Discord, we're a seperate project and don't regularly check this forum or offer support here.

Dear truecharts guru, thanks for your advice, I will bring this discussion to discord.
In the meantime, I could have find a solution to my initial problem to connect with HAProxy load balancer, as follows:
_ On HAProxy: backend set to Encrypt(SSL): NO
_ On nexcloud pod, hpb container, I edited config.php with: 'overwriteprotocol' => 'https',
These two changes allowed me to remove all errors from the admin console.

 

[Robert Thomspon]

Dear truecharts guru, thanks for your advice, I will bring this discussion to discord.
In the meantime, I could have find a solution to my initial problem to connect with HAProxy load balancer, as follows:
_ On HAProxy: backend set to Encrypt(SSL): NO
_ On nexcloud pod, hpb container, I edited config.php with: 'overwriteprotocol' => 'https',
These two changes allowed me to remove all errors from the admin console.

do you use cloudflare for DNS resolution? (and sorry for the delayed response).

ive found that cloudflare while using proxy doesnt play well nwith traefik/haproxy. if you turn off proxy in cloudflare, and set all traffic as https, that should resolve all haproxy issues. (same is said if youre havikng issues with traefik.)

And truecharts is right, thirer stuff is written fairly well to "just woirk" out of the box.

THe problem with their stuff though, is sort of indicated by their last post on this thread... you dont need to know how the apps work, and its sort of feels like thats an intentional way they build their apps (they just work, dont worry about knowing why, or how, or what to do when they break)... Its a good way to build out... if you live in a singular ecosystem and want to function like apple, where the end user doesnt REALLY have any control or know how when it comes to your publications... but, thats all been said before and has been talked about by me ad nauseum.

 

[revoman]

Not sure if this will work for you, but in pfsense I use TLS passthru in HAProxy for my apps that manage their own certificates. See if this helps.

Create a frontend with the interfaces you want to listen on. Include your WAN interface as well (mine is chopped off for security).

Make sure to set the type to "ssl/https(TCP mode)"

At this point there are multiple ways to accomplish the same thing. I'm documenting how I do it currently.

Create a backend that points to the application/service/ip you are trying to access

Then, edit your frontend create an ACL and Action in the frontend that matches the hostname you want to pass thru to

FWIW, my frontend is defined with http-server-close, can't remember why

Save and Apply. You should see your backend in the stats page. The line will turn green once your backend is accessible.

Here is one of my kubernetes clusters.

 

[Robert Thomspon]

Not sure if this will work for you, but in pfsense I use TLS passthru in HAProxy for my apps that manage their own certificates. See if this helps.

Create a frontend with the interfaces you want to listen on. Include your WAN interface as well (mine is chopped off for security).
View attachment 64214

Make sure to set the type to "ssl/https(TCP mode)"
View attachment 64215

At this point there are multiple ways to accomplish the same thing. I'm documenting how I do it currently.

Create a backend that points to the application/service/ip you are trying to access
View attachment 64217
Then, edit your frontend create an ACL and Action in the frontend that matches the hostname you want to pass thru to
View attachment 64216
View attachment 64218

FWIW, my frontend is defined with http-server-close, can't remember why
View attachment 64219

Save and Apply. You should see your backend in the stats page. The line will turn green once your backend is accessible.

View attachment 64220

Here is one of my kubernetes clusters.
View attachment 64222

Does this method play well with CloudFLare using Proxy?(if you happen to use that service)

 

[danb35]

Also you will get B grades on the SSL check because Cloudflare uses internal certs oddly enough and not LetsEncrypt ones for it.

I have no idea where this idea comes from, but it isn't the case. Here's just one example, behind Cloudflare's proxy (you can see in the report it's using their cert), with an A+ grade:

SSL Server Test: wiki.familybrown.org (Powered by Qualys SSL Labs)

 

[Robert Thomspon]

I have no idea where this idea comes from, but it isn't the case. Here's just one example, behind Cloudflare's proxy (you can see in the report it's using their cert), with an A+ grade:

SSL Server Test: wiki.familybrown.org (Powered by Qualys SSL Labs)

I missed that message.

And, yeah, i agree with danb, cloudlfare and traefik combined give me an A+

if youre getting a different result @nubian122, you may want to check your settings from beginning to end.

(here is my A+ certification, and i dont use Cloudflare Proxy, but i do use cloudflare as a DNS forwarder, so it uses CLoudflare's cert)

SSL Server Test (Powered by Qualys SSL Labs)

A comprehensive free SSL test for your public web servers.

www.ssllabs.com

 

[danb35]

i dont use Cloudflare Proxy

Cloudflare really isn't relevant to the result, then. If you're using its proxy, then Cloudflare terminates TLS, and it's going to determine your score. If not, then it depends on how your system is set up--if you're forwarding 80/443 to your Traefik instance, then the Traefik configuration controls. In the case of my result posted up-thread, I am using the Cloudflare proxy for that FQDN, and it's that which is controlling the ssllabs result. That is then hitting Caddy on my OPNsense box (OPNsense has HAProxy too, but even without a GUI to configure it, Caddy is far easier for me to use), which is in turn acting as a reverse proxy to Truecharts Traefik, to Wiki.js--it's kind of complicated. But the point is, Cloudflare's proxy gives A+ on ssllabs.com.

Of course, if @nubian122 is talking about a different SSL checker, then we need to specify which checker is being discussed. But I'm pretty sure ssllabs.com is (by far) the most well-known and widely-used test site.

 

[Imslow]

Not sure if this will work for you, but in pfsense I use TLS passthru in HAProxy for my apps that manage their own certificates. See if this helps.

Create a frontend with the interfaces you want to listen on. Include your WAN interface as well (mine is chopped off for security).
View attachment 64214

Make sure to set the type to "ssl/https(TCP mode)"
View attachment 64215

At this point there are multiple ways to accomplish the same thing. I'm documenting how I do it currently.

Create a backend that points to the application/service/ip you are trying to access
View attachment 64217
Then, edit your frontend create an ACL and Action in the frontend that matches the hostname you want to pass thru to
View attachment 64216
View attachment 64218

FWIW, my frontend is defined with http-server-close, can't remember why
View attachment 64219

Save and Apply. You should see your backend in the stats page. The line will turn green once your backend is accessible.

View attachment 64220

Here is one of my kubernetes clusters.
View attachment 64222

Dear @revoman, thank you for sharing your setting parameters. It turns out that I have a very similar set of parameters. One difference is the frontend type. You recommended to set it up to "ssl/https(TCP mode)"; my configurations uses "http/https(offloading)". What would be the main difference introduced by these settings?

 

[yoloeng]

I'm also looking for a fix for this.