Scale shutsdown on its own

[norjms]

My Truenas installation is shutting itself off and I'm not sure why. I'm running TrueNAS-SCALE-23.10.1.3. I'm not sure where to begin troubleshooting if anyone has any ideas please let me know.

/var/log/messages
Feb 1 17:37:13 truenas kernel: kube-bridge: port 6(veth82fa5703) entered disabled state Feb 1 17:37:13 truenas kernel: device veth82fa5703 left promiscuous mode Feb 1 17:37:13 truenas kernel: kube-bridge: port 6(veth82fa5703) entered disabled state Feb 1 17:37:18 truenas kernel: kube-bridge: port 7(veth41549a53) entered disabled state Feb 1 17:37:18 truenas kernel: device veth41549a53 left promiscuous mode Feb 1 17:37:18 truenas kernel: kube-bridge: port 7(veth41549a53) entered disabled state Feb 1 17:37:19 truenas kernel: kube-bridge: port 4(veth1d6df6a9) entered disabled state Feb 1 17:37:19 truenas kernel: device veth1d6df6a9 left promiscuous mode Feb 1 17:37:19 truenas kernel: kube-bridge: port 4(veth1d6df6a9) entered disabled state Feb 1 17:37:19 truenas kernel: kube-bridge: port 5(veth2ab457ee) entered disabled state Feb 1 17:37:19 truenas kernel: device veth2ab457ee left promiscuous mode Feb 1 17:37:19 truenas kernel: kube-bridge: port 5(veth2ab457ee) entered disabled state Feb 1 17:37:19 truenas kernel: kube-bridge: port 1(vethf21e307d) entered disabled state Feb 1 17:37:19 truenas kernel: device vethf21e307d left promiscuous mode Feb 1 17:37:19 truenas kernel: kube-bridge: port 1(vethf21e307d) entered disabled state Feb 1 17:37:19 truenas kernel: kube-bridge: port 2(veth63b9f5b8) entered disabled state Feb 1 17:37:19 truenas kernel: device veth63b9f5b8 left promiscuous mode Feb 1 17:37:19 truenas kernel: kube-bridge: port 2(veth63b9f5b8) entered disabled state Feb 1 17:37:24 truenas kernel: kube-bridge: port 3(veth7275755c) entered disabled state Feb 1 17:37:24 truenas kernel: device veth7275755c left promiscuous mode Feb 1 17:37:24 truenas kernel: kube-bridge: port 3(veth7275755c) entered disabled state Feb 1 17:37:24 truenas syslog-ng[4027]: syslog-ng shutting down; version='3.38.1' Feb 2 07:10:30 truenas syslog-ng[4022]: syslog-ng starting up; version='3.38.1' Feb 1 17:37:24 truenas systemd-journald[675]: Data hash table of /var/log/journal/e9589d657fe74f3895eb9483f9aaf1fd/system.journal has a fill level at 75.0 (8535 of 11377 items, 6553600 file size, 767 bytes per hash table item), suggesting rotation.
Feb 1 17:37:24 truenas systemd-journald[675]: /var/log/journal/e9589d657fe74f3895eb9483f9aaf1fd/system.journal: Journal header limits reached or header out-of-date, rotating.
Feb 1 17:37:25 truenas kernel: nfsd: last server has exited, flushing export cache Feb 1 17:37:35 truenas systemd-journald[675]: Received SIGTERM from PID 1 (systemd-shutdow).

/var/log/error
Feb 1 17:37:24 truenas blkmapd[3940]: exit on signal(15)

 

[Jorsher]

I recently started having a similar issue on 23.10.1.1 while I was using rsync to move a large amount of data between pools. Around once a day, the server would reboot. A cursory glance at logs didn't show many anything and I just ignored it since it was only a nuisance and the server is getting replaced. I believe it suddenly stopped once I was done shifting data around, but will have to confirm later.

mlkmapd and nfsd are both nfs-related. Are you using NFS services? If so, I might start with that. I'm not using it, so our problems may be unrelated.