APC UPS stops working after some time

[phenx]

So lately I noticed getting warning messages that my UPS (APC Back-UPS CS 650) is not connected any more. So I tried to restart the service. Here's what I got:

Code:

Sep 16 20:13:16 srv-nas notifier: kill: No such process
Sep 16 20:13:31 srv-nas notifier: Driver exited abnormally
Sep 16 20:13:31 srv-nas notifier: Network UPS Tools - UPS driver controller 2.7.3
Sep 16 20:13:31 srv-nas root: /usr/local/etc/rc.d/nut: WARNING: failed precmd routine for nut
Sep 16 20:13:31 srv-nas notifier: /usr/local/etc/rc.d/nut: WARNING: failed precmd routine for nut
Sep 16 20:13:31 srv-nas notifier: Starting nut_upsmon.
Sep 16 20:13:31 srv-nas notifier: kill: No such process
Sep 16 20:13:31 srv-nas notifier: UPS: ups (master) (power value 1)
Sep 16 20:13:31 srv-nas notifier: Using power down flag file /etc/nokillpower
Sep 16 20:13:31 srv-nas upsmon[48209]: UPS [ups]: connect failed: Connection failure: Connection refused
Sep 16 20:13:31 srv-nas upsmon[48209]: Communications with UPS ups lost
Sep 16 20:13:31 srv-nas kernel: pid 48187 (usbhid-ups), uid 66: exited on signal 11
Sep 16 20:13:31 srv-nas notifier: Starting nut_upslog.
Sep 16 20:13:31 srv-nas notifier: Warning: initial connect failed: Connection failure: Connection refused
Sep 16 20:13:32 srv-nas notifier: Stopping nut_upslog.
Sep 16 20:13:32 srv-nas notifier: Waiting for PIDS: 48229.
Sep 16 20:13:32 srv-nas notifier: Stopping nut_upsmon.
Sep 16 20:13:56 srv-nas upsmon[48208]: upsmon parent: read
Sep 16 20:13:56 srv-nas notifier: Waiting for PIDS: 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209, 48209.
Sep 16 20:13:56 srv-nas notifier: nut not running? (check /var/db/nut/upsd.pid).

I then reconnected the UPS:

Code:

Sep 16 20:15:58 srv-nas ugen0.4: <American Power Conversion> at usbus0 (disconnected)
Sep 16 20:16:08 srv-nas ugen0.4: <American Power Conversion> at usbus0

Another attempt to restart the service via web ui was then successful:

Code:

Sep 16 20:16:48 srv-nas notifier: kill: No such process
Sep 16 20:16:48 srv-nas notifier: Using subdriver: APC HID 0.95
Sep 16 20:16:48 srv-nas notifier: Network UPS Tools - Generic HID driver 0.39 (2.7.3)
Sep 16 20:16:48 srv-nas notifier: USB communication driver 0.32
Sep 16 20:16:48 srv-nas notifier: Network UPS Tools - UPS driver controller 2.7.3
Sep 16 20:16:48 srv-nas notifier: Starting nut.
Sep 16 20:16:48 srv-nas notifier: fopen /var/db/nut/upsd.pid: No such file or directory
Sep 16 20:16:48 srv-nas notifier: listening on 127.0.0.1 port 3493
Sep 16 20:16:48 srv-nas notifier: Connected to UPS [ups]: usbhid-ups-ups
Sep 16 20:16:48 srv-nas notifier: /usr/local/etc/nut/cmdvartab not found - disabling descriptions
Sep 16 20:16:48 srv-nas notifier: Starting nut_upsmon.
Sep 16 20:16:48 srv-nas notifier: kill: No such process
Sep 16 20:16:48 srv-nas notifier: UPS: ups (master) (power value 1)
Sep 16 20:16:48 srv-nas notifier: Using power down flag file /etc/nokillpower
Sep 16 20:16:48 srv-nas notifier: Starting nut_upslog.

upsc also seems to read everything just fine after the reconnect:

Code:

upsc ups@localhost
battery.charge: 100
battery.charge.low: 10
battery.charge.warning: 50
battery.date: 2010/10/23
battery.mfr.date: 2010/10/23
battery.runtime: 1200
battery.runtime.low: 120
battery.temperature: 29.2
battery.type: PbAc
battery.voltage: 13.4
battery.voltage.nominal: 12.0
device.mfr: American Power Conversion
device.model: Back-UPS CS 650
device.serial: 4B1043P27416
device.type: ups
driver.name: usbhid-ups
driver.parameter.pollfreq: 30
driver.parameter.pollinterval: 2
driver.parameter.port: /dev/ugen0.4
driver.parameter.synchronous: no
driver.version: 2.7.3
driver.version.data: APC HID 0.95
driver.version.internal: 0.39
input.sensitivity: medium
input.transfer.high: 256
input.transfer.low: 196
input.voltage: 224.0
input.voltage.nominal: 230
output.frequency: 50.0
output.voltage: 230.0
output.voltage.nominal: 230.0
ups.beeper.status: enabled
ups.delay.shutdown: 20
ups.delay.start: 30
ups.firmware: 817.v7 .I
ups.firmware.aux: v7
ups.load: 15.0
ups.mfr: American Power Conversion
ups.mfr.date: 2010/10/23
ups.model: Back-UPS CS 650
ups.productid: 0002
ups.realpower.nominal: 400
ups.serial: 4B1043P27416
ups.status: OL
ups.test.result: No test initiated
ups.timer.reboot: 0
ups.timer.shutdown: -1
ups.timer.start: 0
ups.vendorid: 051d

Now I've faced this multiple times and since the email signalling also does not work for ups (for every other error I do get an email though, any ideas why that might be?) this is preventing me from making this installation reliable :-( I attached a screenshot from my ups configuration.

Any ideas?

Thank you very much

 

[joeschmuck]

You likely have not received an answer yet because you have not provided the basic information we need in order to assist you. Why not provide your system configuration per the forum rules and then see if someone jumps in to help. For us it really helps to understand the hardware components and what version of FreeNAS you are running.

If you want a pure guess off the cuff, I'd say have you tried a different driver, set any tunables, tried a different USB port?

Cheers!

 

[phenx]

Thanks for the hint. I'm in a hurry right now, so information first, I will add this to my signature later

As I'm writing this, the UPS is unavailable again...

Version: FreeNAS-9.3-STABLE-201509022158

Here's my build: Dell PowerEdge T20, Intel Xeon E3-1225 v3 @3.2 GhZ, 20GB RAM, Dell H200 (auf LSI IT-Mode, FW P17). HDDs: 1x 120GB SSD @Mainboard SATA (FreeNAS VM, some other VMs), 2x 3TB WD Red + 2x 6TB WD Red @H200 (RAID1, Datastore + even more VMs). UPS: APC Back-UPS 650.

Tunables: I followed the b3n FreeNAS @ ESXi6 guide so there are tunables for swapfile, vmmemctl_load, vmxnet_load, vmxnet3_load

I tried a different usb port, no changes. Will switch the cable now, though to rule out the basic things first. Will report back if it happens again.

I had tried to use different drivers when I set the UPS up, but non of them worked but the one I use now. Are there any specific other drivers to try out for my UPS?

Cheers!

 

[anodos]

I'd say it's because freenas is in a vm, but that's just a hunch.

 

[DrKK]

I'd say it's because freenas is in a vm, but that's just a hunch.

That's my hunch as well.

 

[SweetAndLow]

Did you pass the USB through to the VM?

 

[joeschmuck]

Wow, look how much help you get just for posting the required info, and no one would have guessed you were running it as a VM. I too agree it's likely a VM issue.

 

[DJ9]

Also, it's more than likely it's a proprietary USB cable that the UPS uses. At least on mine it is.

 

[SweetAndLow]

I will add this to my signature later :)

Careful adding it to your signature. Most people who us a mobile device can't see signatures because the mobile layout is different.

 

[phenx]

This is quite some response, thank you very much!

Also, it's more than likely it's a proprietary USB cable that the UPS uses. At least on mine it is.

You're right. Seems that switching the cable will not be so easy :-/ It's some USB-to-RJ45 variation.

Did you pass the USB through to the VM?

Yes, I did. I'm using the "USB 2.0 and 1.1" controller ESXi 6 offers.
I tried the other offered controller as well (USB 3.0 / xHCI), but that one didn't seem to be recognized by FreeNAS at all.

I switched to another USB Port (USB3 port, where the old one had been a USB2 port) and will see if it happens again..

 

[phenx]

This is quite some response, thank you very much!
I switched to another USB Port (USB3 port, where the old one had been a USB2 port) and will see if it happens again..

And it's disconnected again :-(

I tried using the ESXi USB 3.0 Controller again (with tunable xhci_load this time, so they are enabled) but that only leads to a lot of usb related errors showing up at boot time and the ups not being recognized at all... I could not find any reports of USB3 ESXi working or not working with FreeNAS / FreeBSD on the whole internet. Does anyone have experience with this?

Are there other things I can try (e.g. other drivers that should work) or is it just that this specific configuration (esxi + apc ups) is not meant to work? For what it's worth, I've seen some other users reporting that ups -> passthorugh -> esxi works fine for them.

Is it possible to simulate the "pull cable, plug in cable" procedure with a script that can be run every hour or so?