Service guard extension for SAP - HELP

[Fernando Jose P de Souz]

Hi,

i have a problem with implementation HP Service Guard Extension for SAP.

when i try to run cmrunpkg give me an error.

The package files is in attach.

Wait.

########### Node "SAPNODE1": Starting package at Tue, Jul 7, 2009 06:27:35 PM ###########
Jul 7 06:27:35 PM - Node "SAPNODE1": Activating volume group vg01 with exclusive option.
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c51t0d2":
I/O error
Activated volume group in Exclusive Mode.
Volume group "vg01" has been successfully changed.
Jul 7 06:27:35 PM - Node "SAPNODE1": Checking filesystems:
/dev/vg01/lvol1
/dev/vg01/lvol2
/dev/vg01/lvol3
/dev/vg01/lvol4
/dev/vg01/lvol5
/dev/vg01/lvol6
/dev/vg01/lvol7
/dev/vg01/lvol8
/dev/vg01/lvol9
/dev/vg01/lvol10
/dev/vg01/lvol11
/dev/vg01/lvol12
/dev/vg01/rlvol1:file system is clean - log replay is not required
/dev/vg01/rlvol2:file system is clean - log replay is not required
/dev/vg01/rlvol3:file system is clean - log replay is not required
/dev/vg01/rlvol4:file system is clean - log replay is not required
/dev/vg01/rlvol5:file system is clean - log replay is not required
/dev/vg01/rlvol6:file system is clean - log replay is not required
/dev/vg01/rlvol7:file system is clean - log replay is not required
/dev/vg01/rlvol8:file system is clean - log replay is not required
/dev/vg01/rlvol9:file system is clean - log replay is not required
/dev/vg01/rlvol10:file system is clean - log replay is not required
/dev/vg01/rlvol11:file system is clean - log replay is not required
/dev/vg01/rlvol12:file system is clean - log replay is not required
Jul 7 06:27:35 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol1 at /oracle/P01
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol2 at /usr/sap/P01
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol3 at /sapmnt/P01
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol4 at /oracle/client
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol5 at /oracle/P01/saparch
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol6 at /oracle/P01/flashback
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol7 at /oracle/P01/sapdata6
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol8 at /transfer
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol9 at /NFe
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol10 at /blindagem
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol11 at /archive
Jul 7 06:27:36 PM - Node "SAPNODE1": Mounting /dev/vg01/lvol12 at /ZFI57
Jul 7 06:27:36 PM - Node "SAPNODE1": Adding IP address 180.0.4.97 to subnet 180.0.0.0
+ export MODE=start
+ export SAPSYSTEMNAME=P01
+ PTYPE=sapwas
+ [ Xstart = Xspawn ]
+ + hostname
HNAME=SAPNODE1
+ set -A IP
+ date +%b %e %X
+ echo Jul 7 06:27:36 PM - Node "SAPNODE1": (sapwas_main): Entering SGeSAP start runtime steps ...
Jul 7 06:27:36 PM - Node "SAPNODE1": (sapwas_main): Entering SGeSAP start runtime steps ...
+ date +%b %e %X
+ what /etc/cmcluster/P01/sapwas.cntl
can't open /etc/cmcluster/P01/sapwas.cntl (26)
+ echo Jul 7 06:27:36 PM - Node "SAPNODE1": (sapwas_main):
Jul 7 06:27:36 PM - Node "SAPNODE1": (sapwas_main):
+ CLUS_FILE=/etc/cmcluster.conf
+ get_source /etc/cmcluster.conf critical
Jul 7 06:27:36 PM - Node "SAPNODE1": (get_source): Found /etc/cmcluster.conf
+ CONF_FILE=/etc/cmcluster/P01/sap.conf
+ CON2_FILE=/etc/cmcluster/P01/sapwasP01.conf
+ CON3_FILE=/etc/cmcluster/P01/sap.config
+ CON4_FILE=/etc/cmcluster/P01/sapdbciP01.config
+ FUNC_FILE=/opt/cmcluster/sap/sap.functions
+ FUN2_FILE=/etc/cmcluster/sap.functions
+ CUSO_FILE=/opt/cmcluster/sap/SID/customer.functions
+ CUST_FILE=/etc/cmcluster/customer.functions
+ CUS2_FILE=/etc/cmcluster/P01/customer.functions
+ get_source /etc/cmcluster/P01/sap.conf optional
+ get_source /etc/cmcluster/P01/sapwasP01.conf optional
+ get_source /etc/cmcluster/P01/sap.config optional
+ get_source /etc/cmcluster/P01/sapdbciP01.config optional
+ get_source /opt/cmcluster/sap/sap.functions critical
Jul 7 06:27:36 PM - Node "SAPNODE1": (get_source): Found /opt/cmcluster/sap/sap.functions
+ get_source /etc/cmcluster/sap.functions optional
Jul 7 06:27:37 PM - Node "SAPNODE1": (get_source): Found /etc/cmcluster/sap.functions
+ get_source /opt/cmcluster/sap/SID/customer.functions critical
Jul 7 06:27:37 PM - Node "SAPNODE1": (get_source): Found /opt/cmcluster/sap/SID/customer.functions
+ get_source /etc/cmcluster/customer.functions optional
+ get_source /etc/cmcluster/P01/customer.functions optional
+ FNAME=sapwas_main
+ set -m
+ [ start = start ]
+ initialize
Jul 7 06:27:37 PM - Node "SAPNODE1": (initialize): TRACE POINT
Jul 7 06:27:37 PM - Node "SAPNODE1": (check_versions): TRACE POINT
Jul 7 06:27:37 PM - Node "SAPNODE1": (check_parameters): TRACE POINT
Jul 7 06:27:37 PM - Node "SAPNODE1": (check_parameters): ERROR: Package appears to be specified without virtual IP addresses
Jul 7 06:27:37 PM - Node "SAPNODE1": (check_parameters): ERROR: Check IP[] array in package control file and verify the
Jul 7 06:27:37 PM - Node "SAPNODE1": (check_parameters): ERROR: call syntax in customer_defined_run_commands section
SAPNODE1 /etc/cmcluster/clusterSAP #

[Duncan Edmonstone · Accepted solution]

Fernando,

I see 2 problems here:

- a minor one with some disk device in vg01 not being present (I presume this is an alternate link) - still it needs fixing - is disk mentioned as being part of the VG from a "strings /etc/lvmtab" output?

- some configuration problem where allthough you appear to have an IP in your cntl file, SGeSAP is not finding it.

you appear to be using SGeSAP 5.0, but using legacy rather than modular packages... did you see this note in the admin guide:

----------------------------------
Installation Step: LS400

Create a Serviceguard package directory. This directory will be used by all packages that belong to the SAP System with a specific SAP System ID .

mkdir -p /etc/cmcluster/
----------------------------------

That's here:

http://docs.hp.com/en/T2803-90013/ch03s04.html

You don't appear to have done this - your package seems to be in a directory called something like ClusterSAP... rather than P01

Fix this first and see if that helps...

HTH

Duncan

[Turgay Cavdar]

Hi Fernando,
What about sap.config file , did you change anything there?

Please check the parameters in sap.config file:
NFSRELOC
DBRELOC
CIRELOC

[Fernando Jose P de Souz]

duncan,

imade changes, but get the same error.

the files is in attach.

Fernando.

[Turgay Cavdar]

In your last control script (/etc/cmcluster/P01/P01.control.script) IP, subnet values are not defined:

#IP[0]=""
#SUBNET[0]=""

[Steven E. Protter]

Shalom,

Have you made sure your lock disk is valid?

Have you done the SG reconfiguration plan:

cmquerycl
cmcheckconf
cmapplyconf

SEP

[Fernando Jose P de Souz]

turgay cavdar,

CIRELOC=180.0.4.97
NFSRELOC=180.0.4.97
DBRELOC=180.0.4.97

This Ip is the package

[Turgay Cavdar]

Dont you need to define 180.0.4.97 in /etc/cmcluster/P01/P01.control.script?

#IP[0]=""
#SUBNET[0]=""

If you change your config files can we see last files with cntl log file?

[Doug O'Leary]

Hey;

In addition to all the other things people are telling you, check to ensure you're calling the sapwas.cntl script correctly:

. /etc/cmcluster/SID/sapwas.cntl start SID

and

. /etc/cmcluster/SID/sapwas.cntl stop SID

Notice the '.' in front of the command. When we first implemented the updated sap extensions, we neglected that '.' and got very similar errors until someone actually read the docs...

HTH;

Doug O'Leary

[John Bigg]

You have two problems. Firstly, you have defined NFSRELOC, DBRELOC and CIRELOC to be 180.0.4.97 but this is not defined in the package.

The SGeSAP scripts use the package IP to determine what sort of package it is. If there is no IP address then it does not know it is to be a DB or a CI or and NFS package.

In fact the package has no IP address:

#IP[0]=""
#SUBNET[0]=""

You need to define this to be 180.0.4.97 at which point the package will startup, add this IP address to the system and will then become an NFS, DB and CI package.

The lack of this explains the error:

Jul 7 06:27:37 PM - Node "SAPNODE1": (check_parameters): ERROR: Package appears to be specified without virtual IP addresses

The second problem is that you miss a madatory patch. From the release notes for SGeSAP 5, which I can see you run it states:

Patches
It is mandatory to install the following patch before configuring SGeSAP B.05.00 legacy packages.
à on HP-UX 11.23: PHSS_39357
à on HP-UX 11.31: PHSS_39358
The patch fixes the defects described in
à QXCR1000897415

This CR, it matches your problem too. The title of the CR is:

"check_parameters didn't receive virtual IP information"

and the body of the CR describes the problem you see.

I know you have not got the patch since your files do not show it, but also, the patch is not available yet. It should be available this month.

Until it is available, if you need a legacy package you will need to use SGeSAP 4.51 rather than 5.0

[Fernando Jose P de Souz]

Hi guys,

i solved this problem. see below:


Assigning a different subnet mask with a relocatable IP
Assigning a different subnet mask with a relocatable IP
DocId: UXSGKBRC00003824 Updated: 4/4/06 9:56:00 AM
PROBLEM
Updated 4/4/2006.
I use a custom (non-standard) network configuration. The cmmodnet command in
the package control script assigns the relocatable IP in such a way that the
firewall is no longer pingable. How can I work around this?


CONFIGURATION
lan0 stationary IP: 198.235.80.10 (Class C)
lan0 relocatable IP: 198.235.80.13
SUBNET_MASK: 255.255.255.128 (2 networks with 128 IPs per network)
BROADCAST_ADDRESS: 198.235.80.127
ROUTE_GATEWAY: 198.235.80.2

This configuration violates [http://rfc.net/rfc950.html] RFC950 (Internet Standard Subnetting Procedure)
and [http://rfc.net/rfc1122.html] RFC1122 (Requirements for Internet Hosts -- Communication Layers).

To get around this, I edit the /etc/rc.config.d/nddconf file:

TRANSPORT_NAME[0]=ip
NDD_NAME[0]=ip_check_subnet_addr
(Control the subnet portion of a host address)
NDD_VALUE[0]=0

HP-UX pings gateways for active return when multiple default gateways
are configured. When a gateway is a firewall, no ping is returned.
To overcome this issue, the following lines must be added to the
/etc/rc.config.d/nddconf file:

TRANSPORT_NAME[1]=ip
NDD_NAME[1]=ip_ire_gw_probe
(Enable dead gateway probes)
NDD_VALUE[1]=0



RESOLUTION


There are many configurations that are not officially supported by HP because
they have not been tested. However, the configuration may work under
specific circumstances.

In this case, add or remove IP addresses with different subnet masks using
ifconfig in the customer_defined_run_commands() section of the
package control script. Be aware however that Serviceguard will not transfer
these externally assigned addresses during a local LAN failover! So it is
not just unsupported, it breaks if local lan failover is desired.

This article documents a real situation. The customer wanted to split a
SUBNET (198.235.80) into production and development IP's with no interaction
between them. This violates two network RFC's (RFC950 and RFC1122).
cmmodnet would break the desired routing when a relocatable IP was assigned
via the standard package control script method.

The resolution that follows is provided solely as a successful implementation
technique.

cmmodnet does not use the customers' desired subnet mask when adding a
relocatable IP to a LAN card. When cmmodnet assigns a relocatable IP via the
package control script, it uses the default netmask matching the fixed IP of
the NIC. This IP/netmask is applied to the servers's routing table instead
of the customers' desired netmask (255.255.255.128 in this case). The effect
is that the desired traffic is blocked.

The resolution is to remove the SUBNET parameter in the package configuration
file and IP/SUBNET parameters in the package control script, and to add
ifconfig lines to the customer_defined_run_cmds and
customer_defined_halt_cmds sections.

Package configuration file:
# SUBNET - leave it commented out

Package control script:
#IP[0]=198.235.80.13 #comment out
#SUBNET[0]=198.235.80.0 #comment out

Then add these lines:

function customer_defined_run_cmds
{
# ADD customer defined run commands.
#: do nothing instruction, because a function must contain some command.

ifconfig lan0:1 198.235.80.13 netmask 255.255.255.128 up

test_return 51
}


function customer_defined_halt_cmds
{
# ADD customer defined halt commands.
#: do nothing instruction, because a function must contain some command.

ifconfig lan0:1 0.0.0.0 down

test_return 52
}