adrianwi
Guru
- Joined
- Oct 15, 2013
- Messages
- 1,231
I'm not really sure where to start with this, as the problems I'm having are intermittent, as sometimes I can access the services and other times I can't. I'm struggling to even pin down exactly what or where the problem is, and while I may have developed some skills setting up stuff on FreeNAS, my skills to diagnose problems are exhausting fast :(
I've just built the new system in my signature and migrated everything across and initially, everything appeared to work fine. In summary, I have 9 jails running a variety of services with some (WordPress, Nextcloud, emby, Calibre & Limesurvey) are exposed externally through a jail running NGINX as a reverse-proxy and generating SSL certificates using Letsencrypt (this jail is ssl-proxy). They all have certificates in the format service.domain.co.uk and only connect via https on port 443. I've also built a few Ubuntu VMs, 2 of which are running ONLYOFFICE Document Server and Docker, with a number of containers exposed externally in a similar way to the jails (another Wordpress for testing, Mattermost, Collabra and other things I might be playing with) so always with a SSL certificate and reverse-proxying from 443 to the IP:port in the container.
I noticed the issue with the VMs first, and initially thought it was just the VMs I was having an issue with and not the jails. I'm now not so sure. I never have problems accessing the same services using the IP addresses inside my network, so I think the problem is somewhere from my router to the services. The A records are set up exactly the same as they were on the old machine to the same fixed IP address, so I don't think the issue is getting to my door. When I get the 502/504 error, the certificate icon is showing in the browser, which is making me think it's talking to the ssl-proxy jail which presents the certificate. So the problem seems to be from the ssl-proxy jail to the other jails and VMs. The only thing I've changed with the NGINX.conf files are some of the IP addresses and I've added an example below (for ONLYOFFICE Document Server, which is in a Ubuntu 18.04 VM with nothing else, and the one I first noticed the issue with). They are not all the same and are specific to the service, but again they used to work all of the time. They are still working now, some of the time, which again makes me think it's not this that's the problem.
I'm just at a bit of a loss as to what I need to check next and how in order to find the problem. If any of the above makes sense (which I accept it might not!) can you offer any advice or point me in the direction of something to read and try. My google searches on this have been less than helpful so far.
Thanks in advance.
Adrian
I've just built the new system in my signature and migrated everything across and initially, everything appeared to work fine. In summary, I have 9 jails running a variety of services with some (WordPress, Nextcloud, emby, Calibre & Limesurvey) are exposed externally through a jail running NGINX as a reverse-proxy and generating SSL certificates using Letsencrypt (this jail is ssl-proxy). They all have certificates in the format service.domain.co.uk and only connect via https on port 443. I've also built a few Ubuntu VMs, 2 of which are running ONLYOFFICE Document Server and Docker, with a number of containers exposed externally in a similar way to the jails (another Wordpress for testing, Mattermost, Collabra and other things I might be playing with) so always with a SSL certificate and reverse-proxying from 443 to the IP:port in the container.
I noticed the issue with the VMs first, and initially thought it was just the VMs I was having an issue with and not the jails. I'm now not so sure. I never have problems accessing the same services using the IP addresses inside my network, so I think the problem is somewhere from my router to the services. The A records are set up exactly the same as they were on the old machine to the same fixed IP address, so I don't think the issue is getting to my door. When I get the 502/504 error, the certificate icon is showing in the browser, which is making me think it's talking to the ssl-proxy jail which presents the certificate. So the problem seems to be from the ssl-proxy jail to the other jails and VMs. The only thing I've changed with the NGINX.conf files are some of the IP addresses and I've added an example below (for ONLYOFFICE Document Server, which is in a Ubuntu 18.04 VM with nothing else, and the one I first noticed the issue with). They are not all the same and are specific to the service, but again they used to work all of the time. They are still working now, some of the time, which again makes me think it's not this that's the problem.
Code:
server {
listen 443 ssl;
ssl on;
server_name onlyedit.domain.co.uk;
ssl_certificate /usr/local/etc/letsencrypt/live/onlyedit.domain.co.uk/fullchain.pem;
ssl_certificate_key /usr/local/etc/letsencrypt/live/onlyedit.domain.co.uk/privkey.pem;
add_header Referrer-Policy "no-referrer";
add_header X-Frame-Options "ALLOW-FROM https://nextcloud.domain.co.uk/" always;
add_header X-XSS-Protection "1; mode=block";
add_header X-Robots-Tag none;
add_header X-Download-Options noopen;
add_header X-Permitted-Cross-Domain-Policies none;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload;" always;
# add_header X-Frame-Options SAMEORIGIN;
add_header X-Content-Type-Options nosniff;
location /favicon.ico {
return 204;
access_log off;
log_not_found off;
}
location / {
proxy_pass https://192.168.168.41;
proxy_redirect off;
proxy_read_timeout 1800;
proxy_connect_timeout 1800;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
location /.well-known {
root /usr/local/www;
}
}
I'm just at a bit of a loss as to what I need to check next and how in order to find the problem. If any of the above makes sense (which I accept it might not!) can you offer any advice or point me in the direction of something to read and try. My google searches on this have been less than helpful so far.
Thanks in advance.
Adrian
