Growing problem with PKG

[itsryan]

Over the last few months I have been noticing a problem with some jails. Randomly jails will stop connecting to pkg.freebsd.org only when a command from PKG is used. I am not able to run ANY PKG command or i will get the errors.

root@plex:~ # pkg
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest, please wait...
pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/Latest/pkg.txz: Not Found
A pre-built version of pkg could not be found for your system.
Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'.

root@plex:~ # ping pkg.freebsd.org
PING pkgmir.geo.freebsd.org (96.47.72.71): 56 data bytes
64 bytes from EDIT for privacy: icmp_seq=0 ttl=50 time=32.136 ms
64 bytes from EDIT for privacy: icmp_seq=1 ttl=50 time=31.410 ms
^C
--- pkgmir.geo.freebsd.org ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 31.410/31.773/32.136/0.363 ms

I have looked all over for a solution to this and it keeps coming up as a DNS problem, so I have played around with /etc/resolv.conf making sure I have 3 nameservers 192.168.1.1, 1.1.1.1, 8.8.8.8.
This started on a few jails but its growing to other ones. now it has reached my plex server and I can no longer do updates. but the plex and websites are still accessible for outside/inside the network. its only updates.

root@plex:~ # pkg-static update
pkg-static: Warning: Major OS version upgrade detected. Running "pkg bootstrap -f" recommended
No active remote repositories configured.
root@plex:~ # pkg bootstrap -f
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest, please wait...
pkg: Error fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/Latest/pkg.txz: Not Found
A pre-built version of pkg could not be found for your system.
Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'.

Other jails will sometimes give me this error

root@prowlarr:~ # pkg update
Updating FreeBSD repository catalogue...
pkg: An error occured while fetching package
pkg: An error occured while fetching package
repository FreeBSD has no meta file, using default settings
pkg: An error occured while fetching package
pkg: An error occured while fetching package
Unable to update repository FreeBSD
Error updating repositories!

if i try and update in debug i get this error

root@prowlarr:~ # pkg -d update
DBG(1)[40611]> pkg initialized
Updating FreeBSD repository catalogue...
DBG(1)[40611]> PkgRepo: verifying update for FreeBSD
DBG(1)[40611]> PkgRepo: need forced update of FreeBSD
DBG(1)[40611]> Pkgrepo, begin update of '/var/db/pkg/repo-FreeBSD.sqlite'
DBG(1)[40611]> Request to fetch http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/meta.conf
DBG(1)[40611]> curl_open
DBG(1)[40611]> Fetch: fetcher used: http
DBG(1)[40611]> curl> fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/meta.conf

DBG(1)[40611]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pkgmir.geo.freebsd.org in the .netrc file; using defaults
* Trying 96.47.72.71:80...
* Connected to pkgmir.geo.freebsd.org (96.47.72.71) port 80
> GET /FreeBSD:12:amd64/latest/meta.conf HTTP/1.1
Host: pkgmir.geo.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
DBG(1)[40611]> CURL> attempting to fetch from , left retry 2

* Couldn't find host pkg0.fmt.freebsd.org in the .netrc file; using defaults
* Trying 192.158.248.8:80...
* Connected to pkg0.fmt.freebsd.org (192.158.248.8) port 80
> GET /FreeBSD:12:amd64/latest/meta.conf HTTP/1.1
Host: pkg0.fmt.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
DBG(1)[40611]> CURL> attempting to fetch from , left retry 1

* Couldn't find host pkg0.fra.freebsd.org in the .netrc file; using defaults
* Trying 147.28.184.43:80...
* Connected to pkg0.fra.freebsd.org (147.28.184.43) port 80
> GET /FreeBSD:12:amd64/latest/meta.conf HTTP/1.1
Host: pkg0.fra.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
pkg: An error occured while fetching package
DBG(1)[40611]> Request to fetch http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/meta.txz
DBG(1)[40611]> curl_open
DBG(1)[40611]> Fetch: fetcher used: http
DBG(1)[40611]> curl> fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/meta.txz

DBG(1)[40611]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pkgmir.geo.freebsd.org in the .netrc file; using defaults
* Hostname pkgmir.geo.freebsd.org was found in DNS cache
* Trying 96.47.72.71:80...
* Connected to pkgmir.geo.freebsd.org (96.47.72.71) port 80
> GET /FreeBSD:12:amd64/latest/meta.txz HTTP/1.1
Host: pkgmir.geo.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
DBG(1)[40611]> CURL> attempting to fetch from , left retry 2

* Couldn't find host pkg0.fmt.freebsd.org in the .netrc file; using defaults
* Hostname pkg0.fmt.freebsd.org was found in DNS cache
* Trying 192.158.248.8:80...
* Connected to pkg0.fmt.freebsd.org (192.158.248.8) port 80
> GET /FreeBSD:12:amd64/latest/meta.txz HTTP/1.1
Host: pkg0.fmt.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
DBG(1)[40611]> CURL> attempting to fetch from , left retry 1

* Couldn't find host pkg0.fra.freebsd.org in the .netrc file; using defaults
* Hostname pkg0.fra.freebsd.org was found in DNS cache
* Trying 147.28.184.43:80...
* Connected to pkg0.fra.freebsd.org (147.28.184.43) port 80
> GET /FreeBSD:12:amd64/latest/meta.txz HTTP/1.1
Host: pkg0.fra.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
pkg: An error occured while fetching package
repository FreeBSD has no meta file, using default settings
DBG(1)[40611]> Request to fetch http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/packagesite.pkg
DBG(1)[40611]> curl_open
DBG(1)[40611]> Fetch: fetcher used: http
DBG(1)[40611]> curl> fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/packagesite.pkg

DBG(1)[40611]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pkgmir.geo.freebsd.org in the .netrc file; using defaults
* Hostname pkgmir.geo.freebsd.org was found in DNS cache
* Trying 96.47.72.71:80...
* Connected to pkgmir.geo.freebsd.org (96.47.72.71) port 80
> GET /FreeBSD:12:amd64/latest/packagesite.pkg HTTP/1.1
Host: pkgmir.geo.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
DBG(1)[40611]> CURL> attempting to fetch from , left retry 2

* Couldn't find host pkg0.fmt.freebsd.org in the .netrc file; using defaults
* Hostname pkg0.fmt.freebsd.org was found in DNS cache
* Trying 192.158.248.8:80...
* Connected to pkg0.fmt.freebsd.org (192.158.248.8) port 80
> GET /FreeBSD:12:amd64/latest/packagesite.pkg HTTP/1.1
Host: pkg0.fmt.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
DBG(1)[40611]> CURL> attempting to fetch from , left retry 1

* Couldn't find host pkg0.fra.freebsd.org in the .netrc file; using defaults
* Hostname pkg0.fra.freebsd.org was found in DNS cache
* Trying 147.28.184.43:80...
* Connected to pkg0.fra.freebsd.org (147.28.184.43) port 80
> GET /FreeBSD:12:amd64/latest/packagesite.pkg HTTP/1.1
Host: pkg0.fra.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
pkg: An error occured while fetching package
DBG(1)[40611]> Request to fetch http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/packagesite.txz
DBG(1)[40611]> curl_open
DBG(1)[40611]> Fetch: fetcher used: http
DBG(1)[40611]> curl> fetching http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/packagesite.txz

DBG(1)[40611]> CURL> attempting to fetch from , left retry 3

* Couldn't find host pkgmir.geo.freebsd.org in the .netrc file; using defaults
* Hostname pkgmir.geo.freebsd.org was found in DNS cache
* Trying 96.47.72.71:80...
* Connected to pkgmir.geo.freebsd.org (96.47.72.71) port 80
> GET /FreeBSD:12:amd64/latest/packagesite.txz HTTP/1.1
Host: pkgmir.geo.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
DBG(1)[40611]> CURL> attempting to fetch from , left retry 2

* Couldn't find host pkg0.fmt.freebsd.org in the .netrc file; using defaults
* Hostname pkg0.fmt.freebsd.org was found in DNS cache
* Trying 192.158.248.8:80...
* Connected to pkg0.fmt.freebsd.org (192.158.248.8) port 80
> GET /FreeBSD:12:amd64/latest/packagesite.txz HTTP/1.1
Host: pkg0.fmt.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
DBG(1)[40611]> CURL> attempting to fetch from , left retry 1

* Couldn't find host pkg0.fra.freebsd.org in the .netrc file; using defaults
* Hostname pkg0.fra.freebsd.org was found in DNS cache
* Trying 147.28.184.43:80...
* Connected to pkg0.fra.freebsd.org (147.28.184.43) port 80
> GET /FreeBSD:12:amd64/latest/packagesite.txz HTTP/1.1
Host: pkg0.fra.freebsd.org
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

< HTTP/1.1 404 Not Found
* Failed writing header
* Closing connection
pkg: An error occured while fetching package
Unable to update repository FreeBSD
Error updating repositories!

I know the plex jail is giving me a different error, but I believe they are both caused by something.
I have gone through my ubiquity firewall to see if anything in there might be blocking it but no luck.

Can anyone point me in a direction that will hopefully solve this problem.

 

[Apollo]

Have a look at this post:
https://www.truenas.com/community/threads/truenas-core-jail-upgrade-failed.115548/

 

[itsryan]

Have a look at this post:
https://www.truenas.com/community/threads/truenas-core-jail-upgrade-failed.115548/

I did not know the pkg would be removed once the version is to old, after I updated to current version 13.2-RELEASE it is working again. Not really sure why no where else says this but im glad you were able to help.
Thank you.