Server shut down, but why?

[Smithcraft]

When I returned home from work today, the server was off.

There doesn't appear to have been a power outage, as nothing else shows signs of a power outage that would have shut down the server with a UPS.

If it was my Mac I would just pull up the console and see if I could see something in the logs. However, with the server I don't seem to see where I would look up the system logs.

Am I missing something obvious?

SuperMicro X10SL7-F with 9.10.2-U6.

 

[jgreco]

Start off looking in /var/log/messages. You'll want to roll back past the FreeBSD boot banner, so go to the end and then go up until you see something along the lines of

Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.1-STABLE #0 r321665+9902d126c39(freenas/11.1-stable): Tue Aug 21 12:24:37 EDT 2018
root@nemesis.tn.ixsystems.com:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64 amd64

and then look at what happened right before that.

 

[Smithcraft]

Here's what the log (from /var/log/messages shows:

Code:

Jan 16 00:00:00 freenas newsyslog[62831]: logfile turned over due to size>100K
Jan 16 00:00:00 freenas syslog-ng[1501]: Configuration reload request received, reloading configuration;
Jan 16 00:18:16 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s system-product-name
Jan 16 00:18:16 freenas alert.py: [common.pipesubr:66] Popen()ing: /usr/local/sbin/dmidecode -s baseboard-product-name

The next entry is when I restarted the server, hours later.

 

[jgreco]

Well, that's basically nothing meaningful then.

Can you log into the IPMI and see if it recorded anything in the logs?

Is the host set to start up on power loss?

Does the UPS appear to be in good health? Specifically thinking about the case where a bad battery and a self-test conspire to kill power momentarily.

 

[Smithcraft]

Unfortunately, it seems that logging was not enabled in the Maintenance / System Event Log.

The host is not configured to start up when power comes back on.

The UPS should be in good condition, but I'm looking into test methods now.
[edit]The UPS is across the room from the regular computers, so I can't run the test software.[/edit]

 

[jgreco]

Situation suggests possible bad batteries in a UPS combined with a self-test.

Put the NAS into system BIOS (so no data damage occurs) or put some other similarly-sized dummy load on the UPS.

Check your UPS manual for whatever key sequence is needed for a selftest, and do it. You want a test where the load is actually transferred to the batteries for at least several seconds if possible.

Don't be shocked if nothing bad happens, as these things can be intermittent. Of course, it's totally possible that it was something else entirely as well.