SCALE- (and .3) Dashboard & Reporting failing - New Installation


Oct 15, 2014
So I have two new machines, one running TrueNAS-SCALE- and one running TrueNAS-SCALE-

They have been running for about a month now in tandem with two older FreeNas Core boxes.

Everything appeared to be working fine, including all of my dashboards, and then a couple of days ago, the main dashboards on both servers just blinked offline. I see the system information and storage, but CPU, Memory and Network all all erroring out (Error getting chart data) and all reporting has failed on both systems. These were working before with no issues.

All snapshots, replication, etc are all working still. Everything is working fine, just lost what looks to be all of the reporting and dashboard widgets related to reporting.

Things I have tried:
1) Change Browsers (Chrome, Brave and DuckDuckGo) Private and normal sessions - all fail
2) Change desktops (OSx, Ubuntu Desktop, Windows 11) - All fail
3) Rebooted my backup unit - Complete power cycle
4) Upgraded my backup unit from TrueNAS-SCALE- to TrueNAS-SCALE-
5) Retried 1 & 2 above - all still fail

So I am at a loss as to what to try next.

A little more about the units (identical configuration):
SuperMicro SuperServers
2 x Intel Xeon E5-2695 V4 CPUs
100G Mellanox NICs
2 x RAIDZ2 VDEVs (9 x 18TB HC550 12G/s SAS Drives)
200TB Usable
120TB in use