HPE Community
Operating System - HP-UX
1838646 Members
2483 Online
110128 Solutions
New Discussion

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

 
Laurence Oberman
Advisor

‎12-28-2006 10:06 AM

‎12-28-2006 10:06 AM

Problem installling CFS on 11.23 ServiceGuard 11.17

Hello,

HP-UX lancer B.11.23 U ia64 0700275826 unlimited-user license

The Package SG-CFS is attempting to start the cmvxd daemon, but this then core dumps and prevents the rest of the cfscluster config –s from completing, and both nodes then get taken out.

# file core
core: ELF-32 core file - IA64 from 'cmvxd' - received SIGABRT

I am fully patched at 11.23 and my cluster is fully functional from a ServiceGuard point of view.

Dec 28 17:46:00 lancer cmcld[4157]: Request from root on node lancer to start package SG-CFS-pkg

Dec 28 17:46:00 lancer cmcld[4157]: Request from node coronado to enable global switching for package SG-CFS-pkg.

Dec 28 17:46:00 lancer cmcld[4157]: Request from node coronado to start package SG-CFS-pkg on node lancer.

Dec 28 17:46:00 lancer cmcld[4157]: Executing '/etc/cmcluster/cfs/SG-CFS-pkg.sh start' for package SG-CFS-pkg, as service PKG*9729.

Dec 28 17:46:00 lancer CM-CMD[4320]: cmrunserv SG-CFS-vxconfigd >> /etc/cmcluster/cfs/SG-CFS-pkg.log 2>&1 /etc/cmcluster/cfs/SG-CFS-vxconfigd.sh

Dec 28 17:46:01 lancer CM-CMD[4353]: cmrunserv SG-CFS-cmvxpingd >> /etc/cmcluster/cfs/SG-CFS-pkg.log 2>&1 /usr/lbin/cmvxpingd -t 230

Dec 28 17:46:01 lancer CM-CMD[4358]: cmrunserv SG-CFS-cmvxd >> /etc/cmcluster/cfs/SG-CFS-pkg.log 2>&1 /usr/lbin/cmvxd run -s

/var/adm/cmcluster/cmvxd.socket -t 76

Dec 28 17:46:02 lancer cmvxd[4359]: cmvxd running (pid 4359)

Dec 28 17:46:02 lancer cmvxd[4359]: membership state: node 1 running, node 2 running

Dec 28 17:46:02 lancer cmvxd[4359]: cluster status sequence 1: node 1 up, node 2 up

Dec 28 17:46:02 lancer cmvxd[4359]: Acknowledging cluster status sequence 1

Dec 28 17:46:02 lancer cmvxd[4359]: Assertion failed: vxStarted_ (cfs/cmvxd/MembershipManager.cpp, 249)

Dec 28 17:46:02 lancer cmcld[4157]: Service SG-CFS-cmvxd terminated due to a signal(6).

So any ideas would be helpful. I have spent most of the day reading itrc and google but nothing has helped. I also checked for any known bugs. This is a tough week to get help but I have a migration from a huge 256GB memory Marvel Tru64 Cluster looming and I want to be able to test all this on my system in the lab before I can make recommendations around XP storage, Superdome configurations and CVM /CFS and DMP.

Thanks in advance.

Laurence Oberman



0 Kudos
10 REPLIES 10
Sameer_Nirmal
Honored Contributor

‎12-28-2006 11:36 AM

‎12-28-2006 11:36 AM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

What messages do you see in syslog and in
/etc/cmcluster/cfs/SG-CFS-pkg.log ?

Was LLT and GAB started and running while running "cfscluster config -s" ?

Do you mean by saying the both the nodes taken out as TOC?
0 Kudos
Laurence Oberman
Advisor

‎12-28-2006 11:52 AM

‎12-28-2006 11:52 AM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

Hello,
Yes, GAB and LTT had been started. I provided most of the log messages above.
I also noticed a fencing problem but that appeared to use custom fencing and seemed to recover.

The systems were TOC'D.

See below.
----------
Dec 28 17:46:00 lancer CM-CMD[4251]: From node lancer - Invoked callout script /etc/cmcluster_callout/config/cdispatcher on node lancer
Dec 28 17:46:00 lancer CM-CMD[4251]: From node coronado - Invoked callout script /etc/cmcluster_callout/config/cdispatcher on node coronado
Dec 28 17:46:00 lancer CM-CMD[4278]: /usr/sbin/cmrunpkg -v SG-CFS-pkg
Dec 28 17:46:00 lancer CM-CMD[4278]: Request from root on node lancer to start package SG-CFS-pkg
Dec 28 17:46:00 lancer cmcld[4157]: Request from root on node lancer to start package SG-CFS-pkg
Dec 28 17:46:00 lancer cmcld[4157]: Request from node coronado to enable global switching for package SG-CFS-pkg.
Dec 28 17:46:00 lancer cmcld[4157]: Request from node coronado to start package SG-CFS-pkg on node lancer.
Dec 28 17:46:00 lancer cmcld[4157]: Executing '/etc/cmcluster/cfs/SG-CFS-pkg.sh start'
for package SG-CFS-pkg, as service PKG*9729.
Dec 28 17:46:00 lancer CM-CMD[4320]: cmrunserv SG-CFS-vxconfigd >> /etc/cmcluster/cfs/SG-CFS-pkg.log 2>&1 /etc/cmcluster/cfs/SG-CFS-vxconfigd.sh
Dec 28 17:46:01 lancer CM-CMD[4353]: cmrunserv SG-CFS-cmvxpingd >> /etc/cmcluster/cfs/SG-CFS-pkg.log 2>&1 /usr/lbin/cmvxpingd -t 230
Dec 28 17:46:01 lancer CM-CMD[4358]: cmrunserv SG-CFS-cmvxd >> /etc/cmcluster/cfs/SG-CFS-pkg.log 2>&1 /usr/lbin/cmvxd run -s /var/adm/cmcluster/cmvxd.socket -t 76
Dec 28 17:46:02 lancer cmvxd[4359]: cmvxd running (pid 4359)
Dec 28 17:46:02 lancer cmvxd[4359]: membership state: node 1 running, node 2 running
Dec 28 17:46:02 lancer cmvxd[4359]: cluster status sequence 1: node 1 up, node 2 up
Dec 28 17:46:02 lancer cmvxd[4359]: Acknowledging cluster status sequence 1
Dec 28 17:46:02 lancer cmvxd[4359]: Assertion failed: vxStarted_ (cfs/cmvxd/MembershipManager.cpp, 249)
Dec 28 17:46:02 lancer cmcld[4157]: Service SG-CFS-cmvxd terminated due to a signal(6).
Dec 28 17:46:02 lancer vmunix: LLT INFO V-14-1-10483 16-bit cluster ID (35860) set. Updating protocol version from 3.7 to 4.0
Dec 28 17:46:13 lancer vmunix: GAB INFO V-15-1-20026 Port a registration waiting for seed port membership
Dec 28 17:46:14 lancer vmunix: LLT INFO V-14-1-10024 link 0 (lan0) node 1 active
Dec 28 17:46:14 lancer vmunix: LLT INFO V-14-1-10024 link 1 (lan1) node 1 active
Dec 28 17:46:19 lancer vmunix: GAB INFO V-15-1-20036 Port a gen 13c3c01 membership ;12
Dec 28 17:46:23 lancer syslog: VXFEND: unable to connect to CMVXD!
Dec 28 17:46:29 lancer vmunix: GAB INFO V-15-1-20036 Port b gen 13c3c03 membership ;12
Dec 28 17:46:36 lancer vmunix: GAB INFO V-15-1-20036 Port b gen 13c3c04 membership ; 2
Dec 28 17:46:36 lancer vmunix: GAB INFO V-15-1-20038 Port b gen 13c3c04 k_jeopardy ;1
Dec 28 17:46:36 lancer vmunix: GAB INFO V-15-1-20040 Port b gen 13c3c04 visible ;1
Dec 28 17:46:36 lancer vmunix: NOTICE: VXFEN Notice V-11-1-32776 Deleting deferred message e000000162f0a628...
Dec 28 17:46:36 lancer vmunix: WARNING: VXFEN Warning V-11-1-32814 RECONFIG: snap sender
(1) dead
Dec 28 17:46:36 lancer vmunix: NOTICE: VXFEN Notice V-11-1-32781 entering discover state
from 5

Here is one of the nodes being TOC'D
--------------------------------------
Dec 28 17:49:00 lancer vmunix: Halting lancer to preserve data integrity
Dec 28 17:49:00 lancer vmunix: Reason: A crucial package failed
Dec 28 17:49:00 lancer cmcld[4157]: Service PKG*9729 terminated due to an exit(1).
Dec 28 17:49:00 lancer cmcld[4157]: Package SG-CFS-pkg run script exited with NO_RESTART.
Dec 28 17:49:00 lancer cmcld[4157]: Package SG-CFS-pkg run script failed.
0 Kudos
Laurence Oberman
Advisor

‎12-28-2006 12:16 PM

‎12-28-2006 12:16 PM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

Hello,

My reasoning is that the cmvxd code has a bug and aborts leaving a core in /. I do not any way of analyzing the core because I do not think this was compiled with -g.

The assertion fault is on line 249 in the Membershipmanager.cpp fileit seems from the messages below.

Dec 28 17:46:02 lancer cmvxd[4359]: Assertion failed: vxStarted_ (cfs/cmvxd/MembershipManager.cpp, 249)

Dec 28 17:46:02 lancer cmcld[4157]: Service SG-CFS-cmvxd terminated due to a signal(6).

So I guess I will wait for a response from ServiceGuard Engineering, but this is a bad week for me to get help.
I did not take vacation to get this cluster built for testing.

Thanks

Laurence Oberman
0 Kudos
Laurence Oberman
Advisor

‎12-28-2006 12:34 PM

‎12-28-2006 12:34 PM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

Here is the backtrace from the core.

# adb /usr/lbin/cmvxd core
adb> c
_kill + 0x30
raise + 0x30
_abort + 0x190
_ZN5Utils4Exit3dieEPKcNS0_9DieMethodE + 0xf0
_ZN5Utils4Exit10AssertFailENS0_9DieMethodEPKcS3_i + 0x90
_ZN10CmVxDaemon17MembershipManager20proposedVxMembershipEPNS_19VxClusterMembershipE
+ 0x80
_ZN10CmVxDaemon17MembershipManager11newClStatusEPN8SGStatus20SGClusterStatusValueE
+ 0xcf0
_ZN10CmVxDaemon17MembershipManager8newValueEPN8SGStatus13SGStatusValueE +
0x260
_ZN8SGStatus15SGStatusMonitor11sdbCallbackEP19sdbapi_callback_arg +
0x6b0
_Z21genericStatusCallbackP19sdbapi_callback_arg + 0x1d0
cl_sdb_handle_notifications + 0x11a0
cl_local_handle_queued_req + 0x3b0
cl_local_cl_deliver_callbacks + 0x380
_ZN5Lcomm19LocalNodeConnection9fdHandlerEi + 0x140
_ZN12SingleSelect6Select3runEv + 0x13e0
_ZN10CmVxDaemon17MembershipManager3runEv + 0x80
_ZN10CmVxDaemon10CmVXDaemon3runERK6StringS3_i + 0x100
main + 0x1480
main_opd_entry + 0x50
adb>
0 Kudos
Sameer_Nirmal
Honored Contributor

‎12-28-2006 12:49 PM

‎12-28-2006 12:49 PM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

As per this document
http://docs.hp.com/en/B3936-90100/ch05s06.html

SG-CFS-pkg starts LLT and GAB as its one of its the responsibilities. So I guess if LLT and GAB are already started, the package startup couldn't be completed in proper manner. So you can try starting the package by stopping the exisiting LLT and GAB processes.
0 Kudos
Laurence Oberman
Advisor

‎12-28-2006 12:53 PM

‎12-28-2006 12:53 PM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

Hello,

Look carefully at what I sent already.
GAB and LTT are being started by the PKG script. What happens is that the cmvxd just coredumps and does not remain running so the VXfencd also fails and then the TOC takes place.

What is your Jabber name, I will ping you jabber, or you can ping me on:
laurence.oberman@jabber.hp.com

Thanks

Laurence
0 Kudos
Laurence Oberman
Advisor

‎12-28-2006 12:56 PM

‎12-28-2006 12:56 PM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

Here is the tusc, if I try and start it manually. It dies the same way.

# /usr/local/bin/tusc /usr/lbin/cmvxd run -s /var/adm/cmcluster/cmvxd.socket -t 76
execve("/usr/lbin/cmvxd", 0x9ffffffffffff7e0, 0x9ffffffffffff818) = 0 [32-bit]
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7effb000
open("/usr/lib/hpux32/dld.so", O_RDONLY, 0) ...................... = 3
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 52, SEEK_SET) ........................................... = 52
read(3, "\0\0\006\0\0\04 \0\0\04 \0\0\0\0".., 288) ............... = 288
mmap(NULL, 456944, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_FILE|MAP_SHLIB, 3, 0) = 0xc0016000
sysconf(_SC_PAGE_SIZE) ........................................... = 4096
mmap(NULL, 4168, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7eff9000
mmap(0x7eff5000, 12312, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FILE|MAP_SHLIB, 3, 458752)
= 0x7eff5000
close(3) ......................................................... = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7eff3000
audctl(0x9, NULL, NULL, F_OK) .................................... = 0
procxsec(4, -1, 0x7ffff470, 40) .................................. ERR#251 ENOSYS
sysconf(_SC_PAGE_SIZE) ........................................... = 4096
getuid() ......................................................... = 0 (0)
getuid() ......................................................... = 0 (0)
getgid() ......................................................... = 3 (3)
getgid() ......................................................... = 3 (3)
stat("/usr/lib/hpux32/dpd", 0x7ffff420) .......................... = 0
open("/usr/lib/hpux32/dpd", O_RDONLY, 0) ......................... = 3
fcntl(3, F_SETFD, 0) ............................................. = 0
mmap(NULL, 16384, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efec000
getdents(3, 0x7efec028, 8192) .................................... = 80
getdents(3, 0x7efec028, 8192) .................................... = 0
close(3) ......................................................... = 0
open("/usr/lbin/cmvxd", O_RDONLY, 0) ............................. = 3
lseek(3, 404, SEEK_SET) .......................................... = 404
read(3, "\0\0\002\0\0\0, \0\0\0\aH P \0\0".., 60) ................ = 60
close(3) ......................................................... = 0
open("/usr/lib/hpux32/libsgcl.so", O_RDONLY, 0) .................. = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 2914912, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc0c75000
mmap(NULL, 12792, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7efe8000
mmap(0x7efc5000, 142791, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 2916352) = 0x7efc5000
close(3) ......................................................... = 0
open("/etc/opt/resmon/lib/hpux32/libresmon.so.1", O_RDONLY, 0) ... = 3
fstat(3, 0x7ffff3b0) ............................................. = 0
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 543984, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc19a5000
mmap(NULL, 2680, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7eff2000
mmap(0x7efe6000, 46728, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 544768) = 0x7efb9000
munmap(0x7efb9000, 46728) ........................................ = 0
munmap(0x7eff2000, 2680) ......................................... = 0
mmap(NULL, 51832, PROT_READ, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7efb8000
munmap(0x7efb8000, 51832) ........................................ = 0
mmap(0x7efc4000, 2680, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7efc4000
mmap(0x7efb8000, 46728, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_SHLIB,
3, 544768) = 0x7efb8000
close(3) ......................................................... = 0
open("/usr/lib/hpux32/libstd_v2.so.1", O_RDONLY, 0) .............. = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 1707968, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc0a66000
mmap(NULL, 11304, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7eff0000
mmap(0x7efe5000, 42644, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 1769472) = 0x7efad000
munmap(0x7efad000, 42644) ........................................ = 0
munmap(0x7eff0000, 11304) ........................................ = 0
mmap(NULL, 56360, PROT_READ, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7efa8000
munmap(0x7efa8000, 56360) ........................................ = 0
mmap(0x7efb3000, 11304, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7efb3000
mmap(0x7efa8000, 42644, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_SHLIB,
3, 1769472) = 0x7efa8000
close(3) ......................................................... = 0
mmap(NULL, 16384, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efa4000
open("/usr/lib/hpux32/libCsup.so.1", O_RDONLY, 0) ................ = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 323984, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc0886000
mmap(NULL, 24320, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 327680) = 0x7ef9e000
close(3) ......................................................... = 0
open("/usr/lib/hpux32/libm.so.1", O_RDONLY, 0) ................... = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 1380960, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc0122000
mmap(NULL, 3180, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 1441792) = 0x7eff2000
close(3) ......................................................... = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7eff0000
open("/usr/lib/hpux32/libunwind.so.1", O_RDONLY, 0) .............. = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 284896, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc0c07000
mmap(NULL, 2911, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 286720) = 0x7efb7000
close(3) ......................................................... = 0
open("/usr/lib/hpux32/libc.so.1", O_RDONLY, 0) ................... = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 2629312, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc02a0000
madvise(0xc02a0000, 0x281ec0, MADV_NORMAL) ....................... = 0
mmap(NULL, 41256, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS|MAP_SHLIB, -1, 0) = 0x7ef90000
madvise(0x7ef90000, 0xa128, MADV_NORMAL) ......................... = 0
mmap(0x7ef89000, 26548, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 2629632) = 0x7ef89000
madvise(0x7ef89000, 0x67b4, MADV_NORMAL) ......................... = 0
close(3) ......................................................... = 0
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ef9c000
open("/usr/lib/hpux32/libdl.so.1", O_RDONLY, 0) .................. = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 15728, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc0522000
mmap(NULL, 224, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 16384) = 0x7efb6000
close(3) ......................................................... = 0
open("/ha/be/IA-11.23-IC71R/os/usr/lib/hpux32/libm.so.1", O_RDONLY, 0) ERR#2 ENOENT
open("/ha/sg/be/IA-11.23-IC71R/os/usr/lib/hpux32/libm.so.1", O_RDONLY, 0) ERR#2 ENOENT
open("/ha/be/IA-11.23-IC71R/os/usr/lib/hpux32/libm.so.1", O_RDONLY, 0) ERR#2 ENOENT
open("/usr/lib/hpux32/libm.so.1", O_RDONLY, 0) ................... = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
close(3) ......................................................... = 0
open("/usr/lib/hpux32/libc.so.1", O_RDONLY, 0) ................... = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
close(3) ......................................................... = 0
open("/usr/lib/hpux32/libuca.so.1", O_RDONLY, 0) ................. = 3
fstat(3, 0x7ffff340) ............................................. = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 52) ................ = 52
lseek(3, 0, SEEK_SET) ............................................ = 0
read(3, "7fE L F 0102010101\0\0\0\0\0\0\0".., 1024) .............. = 1024
mmap(NULL, 31472, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0) = 0xc0c4d000
mmap(NULL, 312, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 32768) = 0x7ef9b000
close(3) ......................................................... = 0
open("/opt/graphics/OpenGL/lib/hpux32/libogltls.so", O_RDONLY, 0) ERR#2 ENOENT
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ef87000
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ef85000
mmap(NULL, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ef83000
sigsetreturn(NULL, 0x6211988, 48640) ............................. = 0
mmap(NULL, 3352, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ef82000
sysconf(_SC_CPU_VERSION) ......................................... = 768
brk(0x400081c0) .................................................. = 0
brk(0x4000a1b0) .................................................. = 0
brk(0x4000b000) .................................................. = 0
brk(0x4000c000) .................................................. = 0
brk(0x40010000) .................................................. = 0
brk(0x40011000) .................................................. = 0
fcntl(0, F_GETFL, 4649) .......................................... = 133122
brk(0x40012000) .................................................. = 0
brk(0x40016000) .................................................. = 0
brk(0x40017000) .................................................. = 0
brk(0x40018000) .................................................. = 0
brk(0x40019000) .................................................. = 0
open("/usr/lib/nls/loc/hpux32/locales.1/C", O_RDONLY, 0) ......... ERR#2 ENOENT
fcntl(1, F_GETFL, 4649) .......................................... = 133122
fcntl(2, F_GETFL, 2130386296) .................................... = 133122
fcntl(2, F_GETFL, 512) ........................................... = 133122
fcntl(0, F_GETFL, 512) ........................................... = 133122
brk(0x4001e000) .................................................. = 0
fcntl(1, F_GETFL, 1073807900) .................................... = 133122
fcntl(2, F_GETFL, 1073807900) .................................... = 133122
fcntl(2, F_GETFL, 1073807900) .................................... = 133122
umask(077) ....................................................... = 022
brk(0x4001f000) .................................................. = 0
brk(0x4002c000) .................................................. = 0
open("/etc/cmcluster.conf", O_RDONLY, 0666) ...................... = 3
read(3, "# # # # # # # # # # # # # # # # ".., 512) ............... = 512
read(3, "/ s y s l o g / c m v x d . l o ".., 508) ............... = 36
read(3, 0x4001ea24, 508) ......................................... = 0
close(3) ......................................................... = 0
open("/var/adm/syslog/cmvxd.log", O_WRONLY|O_APPEND|O_CREAT, 0666) = 3
lseek(3, 0, SEEK_END) ............................................ = 41682
getpid() ......................................................... = 4459 (4458)
setrlimit(RLIMIT_CORE, 0x7ffff1d0) ............................... = 0
open("/var/adm/syslog/cmvxd.log", O_WRONLY|O_APPEND|O_CREAT, 0666) = 4
time(0x7ffff0b0) ................................................. = 1167357684
brk(0x4002d000) .................................................. = 0
stat("/usr/lib/tztab", 0x7fffef00) ............................... = 0
open("/usr/lib/tztab", O_RDONLY, 01210) .......................... = 5
mmap(NULL, 16266, PROT_READ, MAP_PRIVATE, 5, 0) .................. = 0x7ef7e000
close(5) ......................................................... = 0
lseek(4, 0, SEEK_END) ............................................ = 41682
lseek(4, 0, SEEK_END) ............................................ = 41682
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 41697
lseek(4, 0, SEEK_END) ............................................ = 41697
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 41698
lseek(4, 0, SEEK_END) ............................................ = 41698
write(4, "L o g s e t u p w i t h l ".., 22) ............... = 22
lseek(4, 0, SEEK_END) ............................................ = 41720
lseek(4, 0, SEEK_END) ............................................ = 41720
write(4, "\n", 1) ................................................ = 1
getpid() ......................................................... = 4459 (4458)
open("/var/adm/cmcluster/cmvxd.pid", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 5
write(5, "4 4 5 9 \n", 5) ........................................ = 5
close(5) ......................................................... = 0
open("/etc/default/syslogd", O_RDONLY, 0666) ..................... = 5
brk(0x4002f000) .................................................. = 0
ioctl(5, TCGETA, 0x7fffe980) ..................................... ERR#25 ENOTTY
read(5, "@ ( # ) d e f a u l t . s y s ".., 8192) .............. = 1356
lseek(5, 4294967043, SEEK_CUR) ................................... = 1103
close(5) ......................................................... = 0
open("/dev/log", O_WRONLY|O_NONBLOCK, 0141300) ................... = 5
fcntl(5, F_SETFD, 1) ............................................. = 0
time(0x7ffff140) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 41721
lseek(4, 0, SEEK_END) ............................................ = 41721
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 41736
lseek(4, 0, SEEK_END) ............................................ = 41736
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 41737
lseek(4, 0, SEEK_END) ............................................ = 41737
write(4, "L o g g i n g t o s y s l o ".., 17) ............... = 17
lseek(4, 0, SEEK_END) ............................................ = 41754
lseek(4, 0, SEEK_END) ............................................ = 41754
write(4, "\n", 1) ................................................ = 1
sched_get_priority_max(SCHED_RR) ................................. = 31
sched_get_priority_min(SCHED_RR) ................................. = 0
sched_setscheduler(0, SCHED_RR, 0x7ffff1c0) ...................... = SCHED_HPUX
mlockall(MCL_CURRENT|MCL_FUTURE) ................................. = 0
sigsetreturn(NULL, 0x6211988, 48640) ............................. = 0
sigvec(SIGTERM, 0x7ffff1c0, 0x7ffff1d0) .......................... = 0
getpid() ......................................................... = 4459 (4458)
time(0x7ffff140) ................................................. = 1167357684
time(0x7fffded0) ................................................. = 1167357684
getpid() ......................................................... = 4459 (4458)
sigsetreturn(NULL, 0x6211988, 48640) ............................. = 0
sigaction(SIGPIPE, 0x7fffdf40, 0x7fffdf70) ....................... = 0
write(5, "< 1 4 > D e c 2 8 2 1 : 0 1 ".., 2048) ............. = 2048
sigaction(SIGPIPE, 0x7fffdf70, NULL) ............................. = 0
lseek(4, 0, SEEK_END) ............................................ = 41755
lseek(4, 0, SEEK_END) ............................................ = 41755
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 41770
lseek(4, 0, SEEK_END) ............................................ = 41770
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 41771
lseek(4, 0, SEEK_END) ............................................ = 41771
write(4, "c m v x d r u n n i n g ( p ".., 24) ............... = 24
lseek(4, 0, SEEK_END) ............................................ = 41795
lseek(4, 0, SEEK_END) ............................................ = 41795
write(4, "\n", 1) ................................................ = 1
socket(AF_UNIX, SOCK_STREAM, 0) .................................. = 6
fcntl(6, F_SETFD, 1) ............................................. = 0
bind(6, 0x7ffff130, 94) .......................................... = 0
listen(6, 0) ..................................................... = 0
chmod("/var/adm/cmcluster/cmvxd.socket", 0600) ................... = 0
time(0x7ffff080) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 41796
lseek(4, 0, SEEK_END) ............................................ = 41796
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 41811
lseek(4, 0, SEEK_END) ............................................ = 41811
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 41812
lseek(4, 0, SEEK_END) ............................................ = 41812
write(4, "L i s t e n i n g f o r v x ".., 39) ............... = 39
lseek(4, 0, SEEK_END) ............................................ = 41851
lseek(4, 0, SEEK_END) ............................................ = 41851
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0f0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 41852
lseek(4, 0, SEEK_END) ............................................ = 41852
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 41867
lseek(4, 0, SEEK_END) ............................................ = 41867
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 41868
lseek(4, 0, SEEK_END) ............................................ = 41868
write(4, "S t a t u s 5 S G - C F S - ".., 51) ............... = 51
lseek(4, 0, SEEK_END) ............................................ = 41919
lseek(4, 0, SEEK_END) ............................................ = 41919
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0f0) ................................................. = 1167357684
brk(0x40032000) .................................................. = 0
lseek(4, 0, SEEK_END) ............................................ = 41920
lseek(4, 0, SEEK_END) ............................................ = 41920
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 41935
lseek(4, 0, SEEK_END) ............................................ = 41935
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 41936
lseek(4, 0, SEEK_END) ............................................ = 41936
write(4, "S t a t u s 3 C L U S T E R ".., 67) ............... = 67
lseek(4, 0, SEEK_END) ............................................ = 42003
lseek(4, 0, SEEK_END) ............................................ = 42003
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0f0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 42004
lseek(4, 0, SEEK_END) ............................................ = 42004
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 42019
lseek(4, 0, SEEK_END) ............................................ = 42019
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 42020
lseek(4, 0, SEEK_END) ............................................ = 42020
write(4, "S t a t u s 3 C M _ S T A T ".., 55) ............... = 55
lseek(4, 0, SEEK_END) ............................................ = 42075
lseek(4, 0, SEEK_END) ............................................ = 42075
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 42076
lseek(4, 0, SEEK_END) ............................................ = 42076
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 42091
lseek(4, 0, SEEK_END) ............................................ = 42091
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 42092
lseek(4, 0, SEEK_END) ............................................ = 42092
write(4, "S e t t i n g p r i o 0 c ".., 58) ............... = 58
lseek(4, 0, SEEK_END) ............................................ = 42150
lseek(4, 0, SEEK_END) ............................................ = 42150
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 42151
lseek(4, 0, SEEK_END) ............................................ = 42151
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 42166
lseek(4, 0, SEEK_END) ............................................ = 42166
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 42167
lseek(4, 0, SEEK_END) ............................................ = 42167
write(4, "S t a t u s 3 C M _ S T A T ".., 55) ............... = 55
lseek(4, 0, SEEK_END) ............................................ = 42222
lseek(4, 0, SEEK_END) ............................................ = 42222
write(4, "\n", 1) ................................................ = 1
utssys(0x7efc7aa0, 64, 5) ........................................ = 0
getpid() ......................................................... = 4459 (4458)
time(0x7ffff0e0) ................................................. = 1167357684
brk(0x40034000) .................................................. = 0
ioctl(3, TCGETA, 0x7ffff040) ..................................... ERR#25 ENOTTY
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 46) ............... = 46
utssys(0x7ffff090, 0, 0) ......................................... = 0
sysconf(_SC_CLK_TCK) ............................................. = 100
access("/etc/cmcluster.conf", R_OK) .............................. = 0
open("/etc/cmcluster.conf", O_RDONLY, 0666) ...................... = 7
brk(0x40036000) .................................................. = 0
ioctl(7, TCGETA, 0x7fffdf20) ..................................... ERR#25 ENOTTY
read(7, "# # # # # # # # # # # # # # # # ".., 8192) .............. = 548
brk(0x40037000) .................................................. = 0
brk(0x40038000) .................................................. = 0
brk(0x40039000) .................................................. = 0
brk(0x4003a000) .................................................. = 0
brk(0x4003b000) .................................................. = 0
read(7, 0x40033958, 8192) ........................................ = 0
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 45) ............... = 45
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 40) ............... = 40
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 40) ............... = 40
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 38) ............... = 38
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 48) ............... = 48
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 62) ............... = 62
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 51) ............... = 51
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 64) ............... = 64
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 76) ............... = 76
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 64) ............... = 64
time(0x7fffded0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 41) ............... = 41
close(7) ......................................................... = 0
time(0x7fffec00) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 78) ............... = 78
socket(AF_UNIX, SOCK_STREAM, 0) .................................. = 7
connect(7, 0x7fffecb0, 94) ....................................... = 0
fcntl(7, F_SETFD, 1) ............................................. = 0
setsockopt(7, SOL_SOCKET, SO_KEEPALIVE, 0x7fffec80, 4) ........... = 0
getsockopt(7, SOL_SOCKET, SO_SNDBUF, 0x4002d034, 0x7fffeca0) ..... = 0
getpid() ......................................................... = 4459 (4458)
brk(0x4003d000) .................................................. = 0
sendmsg(7, "\0\0\0\0\0\0\0\0@ \0acd0\0\0\002".., O_RDONLY) ....... = 116
getpid() ......................................................... = 4459 (4458)
select(8, 0x7fffe3f0, NULL, NULL, NULL) .......................... = 1
recv(7, "\0\0\001\0\0\0\f\0\00188", 12, 0) ....................... = 12
brk(0x40047000) .................................................. = 0
recv(7, "\001\001\003\0\09 f68c14\0\0\002".., 392, 0) ............ = 392
brk(0x4004f000) .................................................. = 0
close(7) ......................................................... = 0
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 42916
lseek(4, 0, SEEK_END) ............................................ = 42916
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 42931
lseek(4, 0, SEEK_END) ............................................ = 42931
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 42932
lseek(4, 0, SEEK_END) ............................................ = 42932
write(4, "I n i t i a l i z e d l c o m ".., 17) ............... = 17
lseek(4, 0, SEEK_END) ............................................ = 42949
lseek(4, 0, SEEK_END) ............................................ = 42949
write(4, "\n", 1) ................................................ = 1
brk(0x4005b000) .................................................. = 0
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 42950
lseek(4, 0, SEEK_END) ............................................ = 42950
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 42965
lseek(4, 0, SEEK_END) ............................................ = 42965
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 42966
lseek(4, 0, SEEK_END) ............................................ = 42966
write(4, "O p e n e d l o c a l n o d ".., 24) ............... = 24
lseek(4, 0, SEEK_END) ............................................ = 42990
lseek(4, 0, SEEK_END) ............................................ = 42990
write(4, "\n", 1) ................................................ = 1
time(0x7fffead0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 78) ............... = 78
socket(AF_UNIX, SOCK_STREAM, 0) .................................. = 7
connect(7, 0x7fffeb80, 94) ....................................... = 0
fcntl(7, F_SETFD, 1) ............................................. = 0
setsockopt(7, SOL_SOCKET, SO_KEEPALIVE, 0x7fffeb50, 4) ........... = 0
getsockopt(7, SOL_SOCKET, SO_SNDBUF, 0x4002d034, 0x7fffeb70) ..... = 0
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 43069
lseek(4, 0, SEEK_END) ............................................ = 43069
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 43084
lseek(4, 0, SEEK_END) ............................................ = 43084
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 43085
lseek(4, 0, SEEK_END) ............................................ = 43085
write(4, "c o n n e c t e d t o c m c ".., 18) ............... = 18
lseek(4, 0, SEEK_END) ............................................ = 43103
lseek(4, 0, SEEK_END) ............................................ = 43103
write(4, "\n", 1) ................................................ = 1
time(0x7ffff030) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 151) .............. = 151
time(0x7ffff030) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 87) ............... = 87
brk(0x4005e000) .................................................. = 0
sendmsg(7, "\0\0\0\0\0\0\0\0@ \0adb0\0\0\002".., O_RDONLY) ....... = 132
getpid() ......................................................... = 4459 (4458)
select(8, 0x7fffe460, NULL, NULL, NULL) .......................... = 1
recv(7, "\0\0\001\0\0\0\f\0\0\0D ", 12, 0) ....................... = 12
recv(7, "\001\001\003\0\09 f68c14\0\0\002".., 68, 0) ............. = 68
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 43342
lseek(4, 0, SEEK_END) ............................................ = 43342
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 43357
lseek(4, 0, SEEK_END) ............................................ = 43357
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 43358
lseek(4, 0, SEEK_END) ............................................ = 43358
write(4, "S e t t i n g p r i o 3 0 ".., 64) ............... = 64
lseek(4, 0, SEEK_END) ............................................ = 43422
lseek(4, 0, SEEK_END) ............................................ = 43422
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 43423
lseek(4, 0, SEEK_END) ............................................ = 43423
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 43438
lseek(4, 0, SEEK_END) ............................................ = 43438
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 43439
lseek(4, 0, SEEK_END) ............................................ = 43439
write(4, "S t a t u s 3 C L U S T E R ".., 67) ............... = 67
lseek(4, 0, SEEK_END) ............................................ = 43506
lseek(4, 0, SEEK_END) ............................................ = 43506
write(4, "\n", 1) ................................................ = 1
time(0x7ffff070) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 157) .............. = 157
time(0x7fffefe0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 145) .............. = 145
time(0x7fffefe0) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 76) ............... = 76
sendmsg(7, "\0\0\0\0\0\0\0\0@ \0add0\0\0\002".., O_RDONLY) ....... = 124
getpid() ......................................................... = 4459 (4458)
select(8, 0x7fffe410, NULL, NULL, NULL) .......................... = 1
recv(7, "\0\0\001\0\0\0\f\0\0\0d4", 12, 0) ....................... = 12
brk(0x40064000) .................................................. = 0
recv(7, "\001\001\001\0\09 f68c14\0\0\002".., 212, 0) ............ = 212
select(8, 0x7fffe410, NULL, NULL, NULL) .......................... = 1
recv(7, "\0\0\001\0\0\0\f\0\0\0D ", 12, 0) ....................... = 12
recv(7, "\001\001\003\0\09 f68c14\0\0\002".., 68, 0) ............. = 68
sendmsg(7, "\0\0\0\0\0\0\0\0@ \0ae10\0\0\002".., O_RDONLY) ....... = 44
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 43885
lseek(4, 0, SEEK_END) ............................................ = 43885
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 43900
lseek(4, 0, SEEK_END) ............................................ = 43900
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 43901
lseek(4, 0, SEEK_END) ............................................ = 43901
write(4, "S e t t i n g p r i o 0 c ".., 54) ............... = 54
lseek(4, 0, SEEK_END) ............................................ = 43955
lseek(4, 0, SEEK_END) ............................................ = 43955
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 43956
lseek(4, 0, SEEK_END) ............................................ = 43956
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 43971
lseek(4, 0, SEEK_END) ............................................ = 43971
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 43972
lseek(4, 0, SEEK_END) ............................................ = 43972
write(4, "S t a t u s 5 S G - C F S - ".., 51) ............... = 51
lseek(4, 0, SEEK_END) ............................................ = 44023
lseek(4, 0, SEEK_END) ............................................ = 44023
write(4, "\n", 1) ................................................ = 1
time(0x7ffff030) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 147) .............. = 147
time(0x7ffff030) ................................................. = 1167357684
write(3, "D e c 2 8 2 1 : 0 1 : 2 4 ".., 87) ............... = 87
sendmsg(7, "\0\0\0\0\0\0\0\0@ \0ae0 \0\0\002".., O_RDONLY) ....... = 128
getpid() ......................................................... = 4459 (4458)
select(8, 0x7fffe460, NULL, NULL, NULL) .......................... = 1
recv(7, "\0\0\001\0\0\0\f\0\00198", 12, 0) ....................... = 12
brk(0x4006e000) .................................................. = 0
recv(7, "\001\001\001\0\09 f68c14\0\0\002".., 408, 0) ............ = 408
select(8, 0x7fffe460, NULL, NULL, NULL) .......................... = 1
recv(7, "\0\0\001\0\0\0\f\0\0\0 ", 12, 0) ....................... = 12
recv(7, "\001\001\005\0\0\0\0\0\0\0\0\0\0".., 32, 0) ............. = 32
select(8, 0x7fffe460, NULL, NULL, NULL) .......................... = 1
recv(7, "\0\0\001\0\0\0\f\0\0\0D ", 12, 0) ....................... = 12
recv(7, "\001\001\003\0\09 f68c14\0\0\002".., 68, 0) ............. = 68
sendmsg(7, "\0\0\0\0\0\0\0\0@ \0aep \0\0\002".., O_RDONLY) ....... = 44
time(0x7ffff0d0) ................................................. = 1167357684
time(0x7fffde60) ................................................. = 1167357684
getpid() ......................................................... = 4459 (4458)
sigaction(SIGPIPE, 0x7fffded0, 0x7fffdf00) ....................... = 0
write(5, "< 1 4 > D e c 2 8 2 1 : 0 1 ".., 2048) ............. = 2048
sigaction(SIGPIPE, 0x7fffdf00, NULL) ............................. = 0
lseek(4, 0, SEEK_END) ............................................ = 44258
lseek(4, 0, SEEK_END) ............................................ = 44258
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 44273
lseek(4, 0, SEEK_END) ............................................ = 44273
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 44274
lseek(4, 0, SEEK_END) ............................................ = 44274
write(4, "C o u l d n o t s e t s t ".., 52) ............... = 52
lseek(4, 0, SEEK_END) ............................................ = 44326
lseek(4, 0, SEEK_END) ............................................ = 44326
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0d0) ................................................. = 1167357684
lseek(4, 0, SEEK_END) ............................................ = 44327
lseek(4, 0, SEEK_END) ............................................ = 44327
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 44342
lseek(4, 0, SEEK_END) ............................................ = 44342
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 44343
lseek(4, 0, SEEK_END) ............................................ = 44343
write(4, "S t a t u s 5 S G - C F S - ".., 51) ............... = 51
lseek(4, 0, SEEK_END) ............................................ = 44394
lseek(4, 0, SEEK_END) ............................................ = 44394
write(4, "\n", 1) ................................................ = 1
time(0x7ffff0d0) ................................................. = 1167357684
time(0x7fffde60) ................................................. = 1167357684
getpid() ......................................................... = 4459 (4458)
sigaction(SIGPIPE, 0x7fffded0, 0x7fffdf00) ....................... = 0
write(5, "< 1 4 > D e c 2 8 2 1 : 0 1 ".., 2048) ............. = 2048
sigaction(SIGPIPE, 0x7fffdf00, NULL) ............................. = 0
lseek(4, 0, SEEK_END) ............................................ = 44395
lseek(4, 0, SEEK_END) ............................................ = 44395
write(4, "D e c 2 8 2 1 : 0 1 : 2 4 ", 15) ................... = 15
lseek(4, 0, SEEK_END) ............................................ = 44410
lseek(4, 0, SEEK_END) ............................................ = 44410
write(4, " ", 1) ................................................ = 1
lseek(4, 0, SEEK_END) ............................................ = 44411
lseek(4, 0, SEEK_END) ............................................ = 44411
write(4, "F a i l e d t o m o n i t o ".., 17) ............... = 17
lseek(4, 0, SEEK_END) ............................................ = 44428
lseek(4, 0, SEEK_END) ............................................ = 44428
write(4, "\n", 1) ................................................ = 1
sigprocmask(SIG_UNBLOCK, NULL, 0x7ffff120) ....................... = 0
sigaction(SIGABRT, NULL, 0x7ffff140) ............................. = 0
close(1) ......................................................... = 0
close(3) ......................................................... = 0
sigprocmask(SIG_BLOCK, 0x7ffff120, NULL) ......................... = 0
sigaction(SIGABRT, 0x7ffff140, NULL) ............................. = 0
sigprocmask(SIG_UNBLOCK, 0x7ffff120, NULL) ....................... = 0
getpid() ......................................................... = 4459 (4458)
Received signal 6, SIGABRT, in kill(), [SIG_DFL], no siginfo
kill(6, SIGABRT) ................................................. [entry]
exit(6) [implicit (kill failure)] ................................ WIFSIGNALED(SIGABRT)|WCOREDUMP
0 Kudos
Laurence Oberman
Advisor

‎12-28-2006 01:06 PM

‎12-28-2006 01:06 PM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

Is it maybe a fencing problem:

Check this out:
Dec 28 20:43:05 lancer cmvxd[4042]: Could not start Vxfend monitoring
Dec 28 20:49:17 lancer cmvxd[4223]: Could not start Vxfend monitoring
0 Kudos
Laurence Oberman
Advisor

‎12-28-2006 01:17 PM

‎12-28-2006 01:17 PM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

Here is the cmvxd.log file.

cmvxd.log: END
Dec 28 21:20:51 Logging to syslog
Dec 28 21:20:51 cmvxd running (pid 6077)
Dec 28 21:20:51 Listening for vxfend connection on fd 6
Dec 28 21:20:51 Status 5 SG-CFS-pkg, priority 0, timeout 0, client
Dec 28 21:20:51 Status 3 CLUSTER_STATUS_NAME, priority 30, timeout 76, client cmvxd
Dec 28 21:20:51 Status 3 CM_STATUS_NAME, priority 0, timeout 0, client
Dec 28 21:20:51 Setting prio 0 callback to 0x4002ce90 for 3 CM_STATUS_NAME
Dec 28 21:20:51 Status 3 CM_STATUS_NAME, priority 0, timeout 0, client
Dec 28 21:20:51 [6077] Bound a callback to 11
Dec 28 21:20:51 [6077] SGCONF=/etc/cmcluster
Dec 28 21:20:51 [6077] SGSBIN=/usr/sbin
Dec 28 21:20:51 [6077] SGLBIN=/usr/lbin
Dec 28 21:20:51 [6077] SGLIB=/usr/lib
Dec 28 21:20:51 [6077] SGRUN=/var/adm/cmcluster
Dec 28 21:20:51 [6077] SGAUTOSTART=/etc/rc.config.d/cmcluster
Dec 28 21:20:51 [6077] SGFFLOC=/opt/cmcluster/cmff
Dec 28 21:20:51 [6077] CMSNMPD_LOG_FILE=/var/adm/SGsnmpsuba.log
Dec 28 21:20:51 [6077] CMGMSD_LOG_FILE=/var/adm/cmcluster/cmgmsd/cmgmsd.log
Dec 28 21:20:51 [6077] CMVXD_LOG_FILE=/var/adm/syslog/cmvxd.log
Dec 28 21:20:51 [6077] CMVXD_LOG_LEVEL=2
Dec 28 21:20:51 [6077] Opening socket /var/adm/cmcluster/.cmcld_local_socket.
Dec 28 21:20:51 Initialized lcomm
Dec 28 21:20:51 Opened local node handle
Dec 28 21:20:51 [6077] Opening socket /var/adm/cmcluster/.cmcld_local_socket.
Dec 28 21:20:51 connected to cmcld
Dec 28 21:20:51 [6077] cl_sdb_private_set_callback: type 3 name CM_STATUS_NAME func 7ef881e0 key 00000001 prio 0 to 0 cli Generic SDB Client flags 0x1
Dec 28 21:20:51 [6077] Sending set callback req: prio 0 to 0 client Generic SDB Client
Dec 28 21:20:51 Setting prio 30 callback to 0x4002ce6c for 3 CLUSTER_STATUS_NAME
Dec 28 21:20:51 Status 3 CLUSTER_STATUS_NAME, priority 30, timeout 76, client cmvxd
Dec 28 21:20:51 [6077] Set sync cb, nh 4004e020, type 3, name CLUSTER_STATUS_NAME, func
7ef881e0, key 00000002, prio 30, timeout 76, client cmvxd, flags 0x1
Dec 28 21:20:51 [6077] cl_sdb_private_set_callback: type 3 name CLUSTER_STATUS_NAME func
7ef881e0 key 00000002 prio 30 to 76 cli cmvxd flags 0x1
Dec 28 21:20:51 [6077] Sending set callback req: prio 30 to 76 client cmvxd
Dec 28 21:20:51 Setting prio 0 callback to 0x4002ce48 for 5 SG-CFS-pkg
Dec 28 21:20:51 Status 5 SG-CFS-pkg, priority 0, timeout 0, client
Dec 28 21:20:51 [6077] cl_sdb_private_set_callback: type 5 name SG-CFS-pkg func 7ef881e0
key 00000003 prio 0 to 0 cli Generic SDB Client flags 0x1
Dec 28 21:20:51 [6077] Sending set callback req: prio 0 to 0 client Generic SDB Client
Dec 28 21:20:51 [6077] cb msg received: type 3, name CM_STATUS_NAME, seq 3, prio 0, flags 0x4
Dec 28 21:20:51 [6077] msg for cb client 7ef881e0: type 3, name CM_STATUS_NAME, prio 0
Dec 28 21:20:51 callback with arg 0x40010450, key 1
Dec 28 21:20:51 callback for type 3, name CM_STATUS_NAME, prio 0
Dec 28 21:20:51 membership state: node 1 running, node 2 running
Dec 28 21:20:51 [6077] cb msg received: type 3, name CLUSTER_STATUS_NAME, seq 2, prio 30, flags 0x4
Dec 28 21:20:51 [6077] msg for cb client 7ef881e0: type 3, name CLUSTER_STATUS_NAME, prio 30
Dec 28 21:20:51 callback with arg 0x40010450, key 2
Dec 28 21:20:51 callback for type 3, name CLUSTER_STATUS_NAME, prio 30
Dec 28 21:20:51 SG status: cluster dtv (972459028) has coordinator coronado, sequence 1
Dec 28 21:20:51 cluster status sequence 1: node 1 up, node 2 up
Dec 28 21:20:51 Acknowledging cluster status sequence 1
Dec 28 21:20:51 Current cl status incarn is 1, queue is 0 long:
Dec 28 21:20:51 Assertion failed: vxStarted_ (cfs/cmvxd/MembershipManager.cpp, 249)
# Dec 28 21:23:52 lancer cmcld[5270]: Halting lancer to preserve data integrity

Message from syslogd@lancer at Thu Dec 28 21:23:52 2006 ...
lancer cmcld[5270]: Halting lancer to preserve data integrity
Dec 28 21:23:52 lancer cmcld[5270]: Reason: A crucial package failed

Message from syslogd@lancer at Thu Dec 28 21:23:52 2006 ...
lancer cmcld[5270]: Reason: A crucial package failed
0 Kudos
Laurence Oberman
Advisor

‎12-29-2006 02:03 AM

‎12-29-2006 02:03 AM

Re: Problem installling CFS on 11.23 ServiceGuard 11.17

I found bunch of patches not applied, Thanks to
Xie, Jun (TSG-SMO-GSS-BJ).

0 Kudos
The opinions expressed above are the personal opinions of the authors, not of Hewlett Packard Enterprise. By using this site, you accept the Terms of Use and Rules of Participation.