Power Manager stops periodically

[Rita Li]

I have a rx6600 server connecting to a R3000 UPS connected, running Power Manager 4.2 (Build 5)

From time to time, the Power Manager would stop (since I check for http://:port everyday), then I need to restart the Power Manager service manually by running this command

/sbin/init.d>./DevMan start
Starting UPS Device Manager Service:
EXE : DevManBE
done.

After the Power Manager is running again, when I look at the "Logs" tab -> UPS Data I am able to tell at what time the Power Manager stops, how can I check out what is causing the Power Manager periodcally?

Thanks

Rita

[Mike Shilladay]

Hi Rita,

I am currently looking at a similar problem - I will let you know if I find anything.

Just out of interest what version of HP-UX are you running Power Manager on ?

Mike.

[Rita Li]

Mike,

Nice to get hold with you, my problem is still not fixed, right now I can only passively check if the Power Manager is running okay or not, if not, then I manually restart it

/>uname -a
HP-UX bdcimfg3 B.11.23 U ia64 3681794811 unlimited-user license

"bdcimfg3" is our server name

Regards,
Rita

[Mike Shilladay]

Hi Rita,

I have been digging around a bit and I have found so far that the powermanager software drops a core file. (For our machine that was 3 mins past midnight), just short of 15 hours from startup.

I noticed that all the libraries are PA-RISC 2.0 libraries, so the core dump may have been produced from the ARIES program. The latest patch for 11.31 is PHSS_41099 and for 11.23 is PHSS_41098. The text for the patch does say that core files may be produced under certain circumstances, however not being a programmer I am not certain if the problems identified are applicable to the Power Manager program.

I will see if I can give that a shot and see what happens. Unfortunately I can only try one fix at a time and wait to see if the problem disappears before I can state that I have found the fix.

I will keep you posted.

Mike.

[Dennis Handly]

>I noticed that all the libraries are PA-RISC 2.0 libraries, so the core dump may have been produced from the ARIES program.

I would assume that the corefile is produced by the broken PA executable and Aries is only doing what it should.

You need to debug the corefile, get a stack trace. What signal does it show with "file core"?

[Mike Shilladay]

Hi Rita,

I reconfigured the port that the Power Manager uses away from the default of 80.

I did check to see if anything was using that port before I originally set it up, but something at somepoint overnight was causing the Power Manager to die and drop a core in /opt/DevMan directory. I guess there must be some conflict overnight.

You can reconfigure the Power Manager's port with the /sbin/init.d/DevMan setup. You may want to check your current configuration first to ensure that you know the method of comms currently used. It is only when it prompts for the web communications port that you will need a free port (other than the default of 80).

For our machine the port I used was 1030 - This is still working just fine so it looks good for us. For the record this is based on the Power Manager failing everynight for the last two weeks since install, so the fact that it is still running initially looks goods.

So it doesn't look like there is a need for any patches.

Hope this helps for you Rita. Let me know if it doesn't though.

Mike.


@Dennis - I did try looking at the core dump, the file command gave the best indication of why it failed, though to my humble knowledge it was a little ambiguous

core.DevManBE: core file from 'DevManBE' - received SIGABRT

[Mike Shilladay]

Sorry Rita the web formatted page is maybe not that clear.

To reconfigure use the command:

/sbin/init.d/DevMan setup

When you use your browser, you will need to append a colon and the port number on the end. So for us, using port 1030, put the following into the browser.

http://"MACHINE NAME OR IP ADDRESS":1030

Mike.

[Rita Li]

Mike,

Thanks very much for your detail info

I am able to run

/sbin/init.d>./DevMan setup
Shutting down UPS Device Manager Service:
DevManBE
done.
Do you wish to use SSL to communicate with HP Power Manager? [Yn]:
Please enter a port to use for web communications [443]:
Is your UPS connected to a network? [yN]:
Please select a serial port to search for a UPS device on:
1) /dev/tty2p0
2) /dev/tty3p0
[1]: 2
Looking for serial devices:
Trying: SerialHID
Trying: Pro UPS
Trying: XCP
Found XCP Device (Model HP R3000 XR) at 19200.

Starting UPS Device Manager Service:
EXE : DevManBE
done.

& via IE https://:443 to note that the Power Manager is working okay

port 443 is the port no. given when running the "/sbin/init.d/Devman setup" also from /etc/services I note that this port is not occupied, I will monitor this new run of Power Manager for the next few days to see whether the situation improves (before running the setup, about once every week the PM dies but not always happen on the same day + time of the week)

Also you are correct, I can locate the file /opt/DevMan/core.DevManBE with timestamp corresponds exactly to the last time when PM dies but I don't know how to decode this core file to some readable statements

Thanks

[Dennis Handly]

>I can locate the file /opt/DevMan/core.DevManBE with timestamp corresponds exactly to the last time when PM dies but I don't know how to decode this core file

What does "file /opt/DevMan/core.DevManBE" show?

Once you know the path to the executable:
/usr/ccs/bin/gdbpa executable /opt/DevMan/core.DevManBE
(gdb) bt
(gdb) q

[Rita Li]

/usr/ccs/bin>./gdbpa executable /opt/DevMan/core.DevManBE
HP gdb 5.6.0 for PA-RISC 1.1 or 2.0 (narrow), HP-UX 11.00
and target hppa1.1-hp-hpux11.00.
Copyright 1986 - 2001 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 5.6.0 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.
..executable: No such file or directory.


warning: Unknown symbols for '/opt/DevMan/core.DevManBE'; use the 'symbol-file'
command.
Core was generated by `DevManBE'.
Program terminated with signal 6, Aborted.

#0 0xc7d1e508 in ()
(gdb) bt
#0 0xc7d1e508 in ()
warning: Attempting to unwind past bad PC 0xc7d1e508
#1 0xc7cb622c in ()
#2 0xc7cb622c in ()
(gdb) q

But we should be running UX 11.i, also there are 1. gdb32 2. gdb64 2 files in /usr/ccs/bin but both don't work


/usr/ccs/bin>ls -alp gdb*
-r-xr-xr-x 1 bin bin 5174032 May 10 2007 gdb32
-r-xr-xr-x 1 bin bin 5182224 May 10 2007 gdb64
lrwxrwxrwx 1 bin bin 7 Nov 24 2008 gdbpa -> ./gdb32

[Mike Shilladay]

Hi Rita,

I was trying to find the download page for the latest (and last) version of Power Manager for HP-UX, which is 4.2 Build 6 (Which is what we run). However, the web page I got it from appears to have gone, only Linux and Windows versions are available.

I am not sure whether you will get any benefit from upgrade to that release. The file is too big to attach it here.(9.5MB)

Mike.

[Dennis Handly]

Again:
What does "file /opt/DevMan/core.DevManBE" show?
Once you know the path to the executable:

>usr/ccs/bin>./gdbpa executable /opt/DevMan/core.DevManBE
>..executable: No such file or directory.
>Core was generated by `DevManBE'.
>Program terminated with signal 6, Aborted.

Replace "executable" by the absolute path of the executable.
Perhaps somewhere in /opt/DevMan/*/DevManBE ??
You may need to use find(1).

>but both don't work

They work fine, if you have the correct path.

Since DevManBE aborted with signal 6, you should look for an error message saying why it is aborting.

[Rita Li]

After changing the port from default 80 to 443 by 07/24/2010 16:19, the PM keeps running & dies shortly by 07/24/2010 23:50, leaving a core file at path /opt/DevMan:-

0 -rw-rw-rw- 1 root sys 0 Jul 23 16:14 alarms.trc
16 -r-------- 1 hpsmh users 1925 Jul 23 16:19 DevManBE.ini
16 drwxrwxrwx 5 root sys 8192 Jul 23 16:27 ./
0 drwxr-x--- 2 root sys 96 Jul 23 16:27 certs/
160 -rw-rw-rw- 1 root sys 78000 Jul 23 16:29 Application.log
3088 -rw-rw-rw- 1 root sys 1579588 Jul 23 23:45 DeviceData.log
4608 -rwx--x--x 1 root root 2353116 Jul 23 23:50 core.DevManBE

Even though able to convert the core.DevManBE into some readable statements but I still get no clue to understand what goes wrong

/opt/DevMan>/usr/ccs/bin/gdbpa -c ./core.DevManBE
HP gdb 5.6.0 for PA-RISC 1.1 or 2.0 (narrow), HP-UX 11.00
and target hppa1.1-hp-hpux11.00.
Copyright 1986 - 2001 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 5.6.0 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.

Reading symbols from DevManBE...done.
Core was generated by `DevManBE'.
Program terminated with signal 6, Aborted.


warning: Load module /opt/DevMan/libssl.sl has been stripped


warning: Load module /opt/DevMan/libcrypto.sl has been stripped

#0 0xc7d1e508 in kill+0x10 () from /usr/lib/libc.2
(gdb) bt
#0 0xc7d1e508 in kill+0x10 () from /usr/lib/libc.2
#1 0xc7cb622c in raise+0x24 () from /usr/lib/libc.2
#2 0xc7cfbf50 in abort_C+0x160 () from /usr/lib/libc.2
#3 0xc7cfbfac in abort+0x1c () from /usr/lib/libc.2
#4 0xc72592ac in __gnu_cxx::__verbose_terminate_handler ()
at ../../../../gcc/libstdc++-v3/libsupc++/vterminate.cc:97
#5 0xc7255a10 in __cxxabiv1::__terminate (handler=0x7b03cb60)
at ../../../../gcc/libstdc++-v3/libsupc++/eh_terminate.cc:43
#6 0xc7255a9c in std::terminate ()
at ../../../../gcc/libstdc++-v3/libsupc++/eh_terminate.cc:53
#7 0xc7255c9c in __cxa_throw (obj=0xc7d1e4f8, tinfo=0x0, dest=0x7af6c1cc)
at ../../../../gcc/libstdc++-v3/libsupc++/eh_throw.cc:77
#8 0xc75ad82c in _ZN3dcl9CCommands21ReadUntilTimerExpiredEPvm+0x104 ()
from /opt/DevMan/libXComUPS.sl
#9 0xc7cb622c in raise+0x24 () from /usr/lib/libc.2
#10 0x7af0b42c in ()
#11 0xc7cb622c in raise+0x24 () from /usr/lib/libc.2
(gdb) q

Pls shed me more lights

Thanks,
Rita

[Dennis Handly]

>Even though able to convert the core.DevManBE into some readable statements but I still get no clue to understand what goes wrong

#4 0xc72592ac __gnu_cxx::__verbose_terminate_handler

There should be some message on stderr.

#7 0xc7255c9c __cxa_throw (obj=0xc7d1e4f8, tinfo=0x0, dest=0x7af6c1cc)
#8 0xc75ad82c _ZN3dcl9CCommands21ReadUntilTimerExpiredEPvm+0x104 libXComUPS.sl

This says that ReadUntilTimerExpired got an error and did a throw.
#4 says that someone made a mistake and didn't catch/handle that throw.

[Rita Li]

There should be some message on stderr. >>> where to check out such message? Can't find any corresponding message written to ths syslog.log

#4 says that someone made a mistake and didn't catch/handle that throw. >>> Do you mean someone made a mistake to the Power Manager?

[Dennis Handly]

>where to check out such message? Can't find any corresponding message written to the syslog.log

To stderr, not syslog.log.
You may have to see what "DevMan start" does so you can capture stderr.

>Do you mean someone made a mistake in Power Manager?

Yes. All throws should catches, else it will abort.

[John O'Neill_6]

Hi All,

Sorry for 'Necro'ing' this thread but this is EXACLY what i'm seeing on my RX2660, it's been driving me mad for months, trying to work out what was going on, I even had a HP tech looking into this, so far nobody has been able to tell me WHY the software crashes to a core file.

I finally came to the ITRC to look for a solution, I should have come here months ago and typed in 'DevMan HP UX' into the search bar....stupid me.

Also, is anyone able to confirm whether or not changing the port number sorts the problem out?

One query I do have, does this mean that between the UPS software crashing out and me detecting it has crashed out, we're vulnerable to a power loss failing to shut down the server gracefully? It's not exactly confidence inspiring...

How do I check for a new version of the UPS monitor software?

-John