Solved: This was caused by an unknown patch for Ubuntu based Linux VMs. I'm going through and painstakingly installing 1 line item at a time and testing. Hope to update this later with the exact patch.
Posting this because it was bonkers and super strange and I hope it helps someone down the road.
Shame on me for keeping my stuff patched ;)
So I have a strange one folks. My Pool for storage is totally fine, behavior of the system itself seems fine. However all of a sudden my linux VMs (ubuntu based) have become super unstable to the point they aren't functionally usable. Super strange, I've had these VMs running fine without any issue for a year or 2 through multiple upgrades in the TrueNas software.
System info
TrueNAS-SCALE-22.12.3.3 (upgraded from 22-12-3 as part of troubleshooting)
AMD Ryzen 1600
48gbs of ram (4 dimms all matching)
VM Behavior:
VM Pool is on an NVME drive
Linux VM boots fine, logs in fine, but when the desktop loads it encounters an error with the desktop environment (doesn't matter which, Cinnamon/Matte/etc). Other installed apps such as PIA VPN crash immediately as well.
Troubleshooting steps:
-Deleted and remade the VMs multiple times with multiple different distros. They work on initial boot but once patched fully and rebooted they break like the above.
-Tried spinning up VMs (new pool) on a new NVME - same result
-Tried spinning up VMs (new pool) on a 2.5 sata SSD - same result
-Tried an NVME in a USBc enclosure as well (that was just me grasping) -same result
- 4 sticks of ram in the system. Went through 1and 2 at a time to see if it was a memory issue (unlikely because the Fileshare Pool is fine and OS seems fine). - same result
-updated the bios to most current on the Mobo (just cuz)
Posting this because it was bonkers and super strange and I hope it helps someone down the road.
Shame on me for keeping my stuff patched ;)
So I have a strange one folks. My Pool for storage is totally fine, behavior of the system itself seems fine. However all of a sudden my linux VMs (ubuntu based) have become super unstable to the point they aren't functionally usable. Super strange, I've had these VMs running fine without any issue for a year or 2 through multiple upgrades in the TrueNas software.
System info
TrueNAS-SCALE-22.12.3.3 (upgraded from 22-12-3 as part of troubleshooting)
AMD Ryzen 1600
48gbs of ram (4 dimms all matching)
VM Behavior:
VM Pool is on an NVME drive
Linux VM boots fine, logs in fine, but when the desktop loads it encounters an error with the desktop environment (doesn't matter which, Cinnamon/Matte/etc). Other installed apps such as PIA VPN crash immediately as well.
Troubleshooting steps:
-Deleted and remade the VMs multiple times with multiple different distros. They work on initial boot but once patched fully and rebooted they break like the above.
-Tried spinning up VMs (new pool) on a new NVME - same result
-Tried spinning up VMs (new pool) on a 2.5 sata SSD - same result
-Tried an NVME in a USBc enclosure as well (that was just me grasping) -same result
- 4 sticks of ram in the system. Went through 1and 2 at a time to see if it was a memory issue (unlikely because the Fileshare Pool is fine and OS seems fine). - same result
-updated the bios to most current on the Mobo (just cuz)
