Problem starting package on one node cluster

[Clive Nicholas]

I am re;latively new to HP-UX server administration and need some help please.
After a major failure of our HP9000/K380 server system which normally consists of a 2 node cluster we have purchased one new server (identical equipment on this node) and attempted to re-load the backup DAT.
All went well until I tried to activate the cluster on the single node using:
cmruncl -n budmc0s
The cluster of a single node formed but the package refused to start.
I then edited the package.cntl file as it stated that I must use the command "vgchange -a y" for a single node cluster and not use exclusive mode.
However, now when the package tries to run I see a "HP Openview DM Error aquiring license LS_NO LICENSES_AVAILABLE" message.
Do I need to request a license from HP to get this to work ... even though I am using identical equipment except on one node only ?
... Or am I doing something wrong ?
All help/advice would be very welcome.

[lawrenzo_1]

hello,

Not too experienced with service guard and only configured ths to run on 1 node when wanting a stanby lancard to run if the primary has issues, attached is a procedure given to me by a freind but when I configured this last week on a test server i did:

Installed service guard

cd /etc/cmluster
cmquerycl -v -C cmclconfig.ascii -n

this created a node on my test server.

HTH helps but as I said not too experienced with MC/SC

[Hoang Chi Cong_1]

Have you try to run the cluster first, then run both of two node later?
#cmhaltcl -f
#cmruncl
(don't use: cmruncl -n budmc0s)
Then run the package:
#cmrunpkg -v -n budmc0s

Is there any error?
Check for the log in the pakgage's log.

Regard,
HoangChiCong

[Clive Nicholas]

Thanks for the 2 postings.
As I already have the cluster configuration and package for a two node cluster I believe it should be possible to start the package but with only the one node available.
I will try the suggestion of starting the cluster first then starting the package after for the one remaining node and see what happens in the error log.... if all else fails I can try setting up a one node configuration from scratch ... but I don't want to go down this route if I don't need to.

[lawrenzo_1]

ok if you do create a new node then it will be work making copies of the original config and map files?

HTH

[Hoang Chi Cong_1]

Hi,
For cluster, you need to startup the cluster by manual. Because, when run cmruncl the cluster configuration will be read and check in all node that defined in.
This paragraph that I cut from the Documentation:
<<
Manual Startup of Entire Cluster
A manual startup forms a cluster out of all the nodes in the cluster
configuration. Manual startup is normally done the first time you bring
up the cluster, after cluster-wide maintenance or upgrade, or after
reconfiguration.
Before startup, the same binary cluster configuration file must exist on
all nodes in the cluster. The system administrator starts the cluster in
Serviceguard Manager or with the cmruncl command issued from one
node. The cmruncl command can only be used when the cluster is not
running, that is, when none of the nodes is running the cmcld daemon.
During startup, the cluster manager software checks to see if all nodes
specified in the startup command are valid members of the cluster, are
up and running, are attempting to form a cluster, and can communicate
with each other. If they can, then the cluster manager forms the cluster.
>>

One time I had the same to you and error happen...and I find out that is the reason.
Goodluck to you
Regard,
HoangChiCong

[Rita C Workman · Accepted solution]

...going to assume that the re-load the backup DAT, is that you used your make_recovery tape to ignite this new (identical equipment) server.
If you did, then all your information is there and your server now believes it's the old server. And I will assume that your old server is off/dead and not on the network anywhere !

Now...you said you can get the cluster to form. That is the first hurdle. ok

You said you edited the cntl file...
Then you say the package won't start.
First rem out starting any databases or processes, just see if you can get your mountpoint up. Then go from there.
or
Please post the section of your ~cntl.log file with all the msgs from the failed start, it might help.

Thanks,
Rita

[Clive Nicholas]

OK ... i've now returned package.cntl file to original state as it was when used for a 2 node cluster.
I then entered cmruncl -n budmc0s command and monitored it with cmviewcl -v regularly.
After about 5 minutes of reformiong the cluster it finally ended but the cluster was DOWN.
Therefore i ran cmruncl -n budmc0s once again and the cluster formed quickly.
The cmviewcl -v command revealed that the cluster was then UP and ACTIVE... but the package was still down.
From the attached package.cnt.log file it looks as if the problem is that the "Activation mode requested for the volume group "/dev/vg03" conflicts with configured mode".
This is why I previously have tried editing the package.cntl file ... commenting out the line VGCHANGE = "vgchange -a e -q n" (exclusive activation) and uncommenting the line VGCHANGE = "vgchange -a y".
However, as you can see from the results further up the log file there seems to be an error aquiring the HPOpenView DM license and the whole server reboots itself.
Why, when the server appears identical in its equipment, and when it has been restored from a full backup in the manner described in the above posting is there an error concerning licensing of HPOpenview DM?
I am not even sure at this moment what Openview DM is !!

[Hoang Chi Cong_1]

I have found there is something wrong with your system:
<vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c0t13d0":
The path of the physical volume refers to a device that does not
exist, or is not configured into the kernel.
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c1t13d0":
The path of the physical volume refers to a device that does not
exist, or is not configured into the kernel.
vgchange: Activation mode requested for the volume group "/dev/vg04" conflicts with configured mode.
vgchange: Warning: couldn't query physical volume "/dev/dsk/c0t13d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query physical volume "/dev/dsk/c1t13d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
ERROR: Function activate_volume_group
ERROR: Failed to activate vg04>>
It seems the vg04 had trouble.
You need to check for the physical volume : /dev/dsk/c1t13d0.
Check with "ioscan -fnC disk". Does it match?
And one more: This message: << activation="" mode="" requested="" for="" the="" volume="" group="">> says that the volume group is not flagged as cluster aware (vgchange â c y vg04).
Therefore, you need to shutdown the cluster, then check for the cluster configuration (cmcheckconf) then apply again the cluster configuration (cmapplyconfg) and for all of the packages.

Hope this helps
HoangChiC

[Rajesh SB]

Hi Clive Nicholas,

HP OpenView DM uses Flexlm license manager.
Flexlm license binding lmhostid. lmhostid is server dependent. If you restored full backup on the different server. Then license can not started due mismatch in lmhostid.
This you can verify by
run
# lmutil lmhostid
It returns lmhostid compare with hostid entry in the license file /etc/opt/OV/share/conf/license/DM.lic.

I hope it may help you.

Thanks & Regards,
Rajesh SB

[Clive Nicholas]

Thanks Rajeesh,
However, the lmutil commands do not work on my HP-UX 11 server.
Using your advice however I have discovered that i can see my HostId (8 digit Hex number)if I enter the following command
#echo `/bin/uname -i` 16op|dc
I was unable to find the license folder and DM.lic file you mentioned either.
However, i agree that my problem does indeed sem to be that the HP Openview DM license no longer matches the HostId of the machine I am trying to run on.
I guess the host ID is probably in-built in the processor perhaps ?

Apologies to Jimkery (HoangChiCong) ... I think you may have analysed errors logged in my package.cntl.log file which were very old ... the latest error was at the bottom of the file. The server had been reconfigured since the errors he ponted out to me. The server now has 2 local disks (mirrored) and 6 shared disks installed.
The 6 shared disks are arranged as 3 volume groups.. each group has a mirrored disk .One sid is on one SCSI chain and the other mirrored set on the other SCSI chain.
But it wasn't always like this .. so this is where you saw the earlier errors in the log file.

[Geoff Wild]

You have 2 issues - one is a licensing issue - other is your vg:

You need to do a vgchange -c y /dev/vg03 and make sure vg03 is NOT activated at boot.

Make sure AUTO_VG_ACTIVATE=0 in /etc/lvmrc

In custom_vg_activation() section, add any vg's other then vg00 that are not part of MC/SG

Example:

{
# e.g. /sbin/vgchange -a y -s
# parallel_vg_sync "/dev/vg00 /dev/vg01"
# parallel_vg_sync "/dev/vg02 /dev/vg03"
/sbin/vgchange -a y -s /dev/vg00
# vg01 for mercator
/sbin/vgchange -a y -s /dev/vg01

return 0
}



Rgds...Geoff

[Clive Nicholas]

Thanks for all your help.
I have been able to make sure that the volume groups are now cluster aware.
The only questions I have left are concerning licensing and I will try to get further help from HP on this.
Many thanks,

Best Regards,

Clive