topic Pkackage/Listener issue. in Operating System - Linux

Pkackage/Listener issue.

sandeepbodkhe_1 — Wed, 01 Dec 2010 09:29:58 GMT

Hi , I have 4 node cluster with oracle toolkit working. Recently all of the packages on one node have failed over to another node. when i checked for the reason, i found one error in pkg log of each file. "listener Failure detected "

Fact about environment
-----------------------
1) All listener are using a DNS names for connection and dependant on DNS for name resolution.
2) A DNS entry exist with the package IP
3) DNS connectivity during the failover time was all ok, and even no network related error found anywhere..all other nodes using same thing were all ok.

Event chain.
-------------
1) Listener failure detected. Pkg was up till that. so is Pkg ip and DNS)

2) Sg tried to start the Listener up again. (Dont know why.. or it is default behaviour.)but it should have been able to start that listener successfully because pkg ip by the time was still up.)

3)It failed to start listner again with error.
"Connect failed because target host or object does not exist"

4) Finaly it failed over pkg to another node.

Finding.
==============
1) Even when the all pkg were failed detecting listener failure, they were found up and running in process table. i noticed this when i tried fail back. which failed due to this issue.

2) I then rebooted node and tried failback again which went successful.

Questions i am searching answers for.
-------------------------------------
1)How does the Serviceguard or oracle toolkit. checks for the listener status ? why has it failed to detect running listener.

2) If listener actually failed why is it still lived in process table. when package did not complained about unclean halt..

i am attaching one of the pkg log. for referances.

Re: Pkackage/Listener issue.

John Bigg — Thu, 02 Dec 2010 08:50:17 GMT

I am sure you have seen the lines in the package log showing the failure:

Nov 23 05:41:29 - Node "dfuaor13" Oracle Listener sqldvlm6_listener Failure detected!
Nov 23 05:41:29 root@dfuaor13 master_control_script.sh[20097]: ###### Halting package SQLDVLM6 ######

So you need to determine why this occurred. How the listener is monitored is dependent on the exact version. The best way is to simply search for this message in the toolkit script and see what test failed.

I cannot say since I do not know your exact version.

Re: Pkackage/Listener issue.

sandeepbodkhe_1 — Fri, 03 Dec 2010 10:17:32 GMT

Thanks you for response.

Here are those details.

root@dfuaor13 /.root]$ swlist -l product |grep -i serviceguard
PHSS_40794 1.0 Serviceguard A.11.19.00
SG-Apache-Tool B.05.00 Serviceguard Apache Script Templates
SG-DB2-Tool B.05.00 Serviceguard DB2 Script Templates
SG-Manager A.05.03 Serviceguard Java GUI
SG-Manager-Addon A.05.03 Serviceguard Manager/OpenView Integration Add-on
SG-Mysql-Tool B.05.00 Serviceguard Mysql Script Templates
SG-NFS-Tool A.11.31.05 Serviceguard NFS Script Templates
SG-Oracle-Tool B.05.00 Serviceguard Oracle Script Templates
SG-Samba-Tool B.05.00 Serviceguard Samba Script Templates
SG-Sybase-Tool B.05.00 Serviceguard Sybase Script Templates
SG-Tomcat-Tool B.05.00 Serviceguard Tomcat Script Templates
SGManagerPI B.02.00 HP Serviceguard Manager
SGWBEMProviders A.03.00.00 HP Serviceguard WBEM Providers SD Product
ServiceGuard A.11.19.00 Serviceguard SD Product
[root@dfuaor13 /.root]$ uname -a
HP-UX dfuaor13 B.11.31 U ia64 4224542528 unlimited-user license