Not running package on node in metrocluster serviceguard

[Goriik]

I have two nodes in the cluster. I made two packages. On the Primary Node, these two packages are run. But start at Secondary Node packages fail.


[root@primary_node]# cmrunpkg -n secondary_node pkg1
Running package pkg1 on node secondary_node

The package script for pkg1 failed with no restart. pkg1 should not be restarted
Unable to run package pkg1 on node secondary_node
Check the syslog and pkg log files for more detailed information
cmrunpkg: Unable to start some package or package instances



Log with primary_node

Mar 3 18:05:03 primary_node syslog: cmrunpkg -n secondary_node pkg1
Mar 3 18:05:03 primary_node inetd[14528]: hacl-cfg/tcp: Connection from localhost (127.0.0.1) at Wed Mar 3 18:05:03 2010
Mar 3 18:05:03 primary_node above message repeats 16 times
Mar 3 18:05:03 primary_node inetd[14529]: auth/tcp: Connection from localhost (127.0.0.1) at Wed Mar 3 18:05:03 2010
Mar 3 18:05:03 primary_node inetd[14530]: auth/tcp: Connection from secondary_node (10.1.14.62) at Wed Mar 3 18:05:03 2010
Mar 3 18:05:03 primary_node syslog: Request from root on node primary_node to start package pkg1
Mar 3 18:05:03 primary_node cmcld[2705]: Request from root on node primary_node to start package pkg1

Mar 3 18:07:06 primary_node cmcld[2705]: Request from node secondary_node to disable global switching for package pkg1.
Mar 3 18:07:06 primary_node syslog: Request from root on node primary_node to start package pkg1 failed
Mar 3 18:09:25 primary_node syslog: rm_log_init: fopen of file /etc/opt/resmon/log/client.log failed: Permission denied
Mar 3 18:09:48 primary_node inetd[14755]: hacl-cfg/udp: Connection from secondary_node (10.1.14.62) at Wed Mar 3 18:09:48 2010
Mar 3 18:09:48 primary_node inetd[14756]: hacl-cfg/tcp: Connection from secondary_node (10.1.14.62) at Wed Mar 3 18:09:48 2010

Log with secondary_node

18:05:03 secondary_node inetd[11608]: hacl-cfg/udp: Connection from primary_node (10.1.14.74) at Wed Mar 3 18:05:03 2010
Mar 3 18:05:03 secondary_node inetd[11609]: hacl-cfg/tcp: Connection from primary_node (10.1.14.74) at Wed Mar 3 18:05:03 2010
Mar 3 18:05:03 secondary_node cmcld[2705]: Request from node secondary_node to start package pkg1 on node secondary_node.
Mar 3 18:04:28 secondary_node inetd[1560]: registrar/tcp: Access denied for localhost (127.0.0.1) at Wed Mar 3 18:04:28 2010
Mar 3 18:05:03 secondary_node above message repeats 3 times
Mar 3 18:05:03 secondary_node cmcld[2705]: Executing '/etc/cmcluster/scripts/mscripts/master_control_script.sh start' for package pkg1, as service PKG*31745.
Mar 3 18:05:03 secondary_node cmserviced[2710]: Request to perform run service PKG*31745
Mar 3 18:05:05 secondary_node LVM[11709]: vgchange -a e vgdata_acq
Mar 3 18:05:07 secondary_node syslog: cmmodnet -a -i 10.1.14.65 10.1.14.0
Mar 3 18:05:07 secondary_node su: + tty?? root-acqtest
/dev/vgdata_acq/svbin file system full (8 block extent)
Mar 3 18:05:37 secondary_node su: + tty?? root-acqtest
Mar 3 18:06:10 secondary_node inetd[1560]: registrar/tcp: Access denied for secondary_node (10.1.14.62) at Wed Mar 3 18:06:10 2010
Mar 3 18:06:53 secondary_node syslog: cmmodnet -r -i 10.1.14.65 10.1.14.0

[melvyn burnard]

so what do the package logs show on tthe secondary node?

[Kapil Jha]

let us know what does
etc/cmcluster/scripts/mscripts/pkg_name.log says.

This line seems interesting
Mar 3 18:06:10 secondary_node inetd[1560]: registrar/tcp: Access denied for secondary_node (10.1.14.62) at Wed Mar 3 18:06:10 2010

Do you know probably why this could have happened, you may like to explain some of ur findings.

BR,
Kapil+

[Stephen Doud]

Each package keeps a log of startup and shutdown events on each node. As others have stated, we need to see the package log for "pkg1" on the secondary node.

Use the following command to locate the package log file:
$ cmviewcl -v -f line | grep pkg1 | grep log

The 'access denied for' messages relate to service blocking. Check /var/adm/inetd.sec on each node and insure that identd and cmclconfd are ALLOWED for each IP each server hosts, including the loopback IP.

[kashyapi]

did you add both the nodes in cmnodelist file.copy the .cntl and .conf file from pri node to secondary node oe in the both the node files should be same.also pls make sure files are same in /etc/cmcluster/ for both node including scrtipts.

another thing did you copyied pkg.conf and pkg.cntl file from another environment or you have created #cmmakepkg command?