Service Guard 11.18 installation issue

[Muhammad Ahmad]

Service Guard installation phase completes successfully but one file set has error.

ERROR: Verify failed Cluster-Monitor.CM-CORE,l=/,r=A.11.18.00

Service Guard reinstalled but the same error persist.
What to do?

[sudheerch]

check the swinstall.log for more error information
#more /var/adm/sw/swinstall.log

[AnthonySN]

what is you OS version?
is it patched to the latest QPK?
try using MCSG 11.19

[Muhammad Ahmad]

hi,

as, existing 2nodes-cluster running with dec, 2007 QPK and SG 11.18, that's why we install the same QPK and SG version in the 3rd node (new-cluster-member).

some more details about our findings related to SG installation:
----------------------------------------
ERROR: File "/usr/lib/hpux64/libsgcl.so" should have mtime
"1174003785" but the actual mtime is "1174003784".
ERROR: Fileset "Cluster-Monitor.CM-CORE,l=/,r=A.11.18.00" had file
errors.

* Summary of Analysis Phase:
Verified T1905CA,l=/,r=A.11.18.00
+ Verified openssl.OPENSSL-SRC,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-RUN,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-PVT,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-PRNG,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-MIS,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-MAN,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-LIB,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-INC,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-DOC,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-CONF,l=/opt/openssl,r=A.00.09.07m.003
+ Verified openssl.OPENSSL-CER,l=/opt/openssl,r=A.00.09.07m.003
+ Verified WBEMServices.WBEM-CORE,l=/,r=A.02.07
+ Verified WBEMServices.WBEM-CORE-COM,l=/,r=A.02.07
+ Verified WBEMServices.WBEM-MAN,l=/,r=A.02.07
+ Verified WBEMServices.WBEM-MX,l=/,r=A.02.07
Verified CM-Provider-MOF.CM-MOF,l=/,r=B.05.00.00
Verified CM-Provider-MOF.CM-PROVIDER,l=/,r=B.05.00.00
Verified Cluster-OM.CM-DEN-MOF,l=/,r=B.05.00.00
Verified Cluster-OM.CM-DEN-PROV,l=/,r=B.05.00.00
Verified Cluster-OM.CM-OM,l=/,r=B.05.00.00
Verified Cluster-OM.CM-OM-AUTH,l=/,r=B.05.00.00
Verified Cluster-OM.CM-OM-AUTH-COM,l=/,r=B.05.00.00
Verified Cluster-OM.CM-OM-COM,l=/,r=B.05.00.00
Verified Cluster-OM.CM-OM-MAN,l=/,r=B.05.00.00
Verified Cluster-OM.CM-OM-TOOLS,l=/,r=B.05.00.00
Verified SGWBEMProviders.SGPROV-CORE,l=/,r=A.02.00.00
Verified SGWBEMProviders.SGPROV-DOC,l=/,r=A.02.00.00
Verified SGWBEMProviders.SGPROV-MOF,l=/,r=A.02.00.00
Verified SGManagerPI.SGMGRPI,l=/,r=B.01.01
+ Verified EMS-Core.EMS-CORE-COM,l=/,r=A.04.20.23
+ Verified EMS-Core.EMS-CORE,l=/,r=A.04.20.23
ERROR: Verify failed Cluster-Monitor.CM-CORE,l=/,r=A.11.18.00
Verified Cluster-Monitor.CM-CORE-COM,l=/,r=A.11.18.00
Verified Cluster-Monitor.CM-CORE-MAN,l=/,r=A.11.18.00
Verified Package-CVM-CFS.CM-CVM-CFS,l=/,r=A.11.18.00
Verified Package-CVM-CFS.CM-CVM-CFS-COM,l=/,r=A.11.18.00
Verified Package-Manager.CM-PKG,l=/,r=A.11.18.00
Verified Package-Manager.CM-PKG-COM,l=/,r=A.11.18.00
Verified Package-Manager.CM-PKG-MAN,l=/,r=A.11.18.00
Verified ServiceGuard.CM-SG,l=/,r=A.11.18.00
ERROR: 1 of 40 filesets had Errors.
* 39 of 40 filesets had no Errors or Warnings.
ERROR: The Analysis Phase had errors. See the above output for
details.
----------------------------

Please note, we have tried to do the re-install.
but the same fileset have failed verification.

# swlist |grep -i openssl
OpenSSL A.00.09.07m.003 Secure Network Communications Protocol
# swlist |grep -i eventm
EventMonitoring A.04.20.23 Event Monitoring Service
# swlist|grep -i guard
T1905CA A.11.18.00 Serviceguard


please comment. . .

[sudheerch · Accepted solution]

you can check more errors from /var/adm/sw/swagent.log and let me know the same

and also check the swinstall options before installing, like
1.Autoselect dependencies when marking software
2.Enforce dependency analysis errors in agent
3.Reinstall files even if same one already there

[Dennis Handly]

>ERROR: File "/usr/lib/hpux64/libsgcl.so" should have mtime "1174003785" but the actual mtime is "1174003784".

This is very strange, it is only off by one second.

[Muhammad Ahmad]

Hi sudheerch,

Thanks for your reply.
PFA, /var/adm/sw/swagent.log.
for your information, we did selection in sam the swinstall options, you mentioned.

Hi Dennis Handly,

Thanks for your reply.

>> This is very strange, it is only off by one second.

Can you suggest the solutoin to successfully troubleshoot that issue.

Regards,
Ahmad

[AnthonySN]

how abt copy the file "/usr/lib/hpux64/libsgcl.so" from a working node and reinstall SG.

[Muhammad Ahmad]

Hi Dennis Handly,

Please note:

during o/s 11.23 installation, SG 11.19 was installed (bundled with o/s).

then it was uninstalled and 11.18 was installed from another media.

I hope that information was usefull too for your analysis.

Regards,

[Muhammad Ahmad]

Hi SASJ,

just copied the "/usr/lib/hpux64/libsgcl.so.2" from existing clustered node and re-execute the process.

but no use.

regards,

[likid0]

""just copied the "/usr/lib/hpux64/libsgcl.so.2" ""

the file is complaining about is libsgcl.so, not libsgcl.so.2

I had a similar problem a long time ago, and I remember i fixed it with:

#/usr/sbin/swmodify -x files="/usr/lib/hpux64/libsgcl.so" Cluster-Monitor.CM-CORE-COM

but in my case the problem was running swverify on an installed fileset.

could you also attach or list all the service guard filesets currently installed on the server.

[likid0]

In you case if you have the software in a depot,and not in the system, you could also try with:

#swmodify -d -x files='/usr/lib/hpux64/libsgcl.so' Cluster-Monitor.CM-CORE-COM @ /path/to/yourdepot

checkout swmodify mans page

[Dennis Handly]

>Can you suggest the solution to successfully troubleshoot that issue?

You can just ignore it. This type of difference seems pretty much impossible, unless there is sloppiness in packaging the product.

You could do "swverify -F Cluster-Monitor.CM-CORE".

To actually figure out why, it would probably be better if you just reported this bug to the Response Center.

>SASJ: how about copy the file "/usr/lib/hpux64/libsgcl.so" from a working node and reinstall SG.

Why do you think the copy there isn't working? And if you copied it, it may have the wrong time too.

>just copied the "/usr/lib/hpux64/libsgcl.so.2" from existing clustered node and re-execute the process.

Was the timestamp off by the same numbers?

>Daniel: the file is complaining about is libsgcl.so, not libsgcl.so.2

Ah, what is a timestamp for a symlink.

[Muhammad Ahmad]

Hi All,

Big Round of Applause for You.

The fileset issue is resolved, by copying both the fileset plus symbolic link, and retrying the operation.

in our understanding, during SG 11.19 to SG 11.18 downgrade activities, HPUX did'nt manage this thing properly, and finally we need to do the manual copy paste of the liberary file from running nodes.

Please comment.

Secondly, just a quick question regarding running "cmquerycl", please have look to given below scenarios:

1. # cmquerycl -n

>>> allowed, results display on output screen

2. # cmquerycl -n

>>> allowed, results display on output screen

3. # cmquerycl -n

>>> Not-Allowed, thows given below errors:


Permission denied to
Warning: Unable to determine local domain name for
Node is refusing Serviceguard communication.
Please make sure that the proper security access is configured on node
ihasde01 through either file-based access (pre-A.11.16 version) or role-based
access (version A.11.16 or higher) and/or that the host name lookup
on node ihasde01 resolves the IP address correctly.
Failed to gather configuration information.

++ In our understanding, this is a normal behaviour, As a Non-Clusterd-Node which is Currently not the part of a RUNNING cluster CanNot and ShouldNot allowed to execute cmquerycl.

As 2 nodes cluster is running, any other node with SG (same version installed) and OS (same version installed) is not allowed to run cmquerycl (step 3. above) in our understanding.

Please Comment.

Regards,
Ahmad

[AnthonySN]

http://www.olearycomputers.com/ll/mcsg/mcsg_mknode.html

Obtain a current cluster configuration file
cmgetconf current_config
Create a generic configuration file
cmquerycl -v -C generic_config -n ${node1} -n ${node2} -n ${node3}
NOTE: Ensure the generic_config file is different from the current_config
Merge the two configuration files.
Cluster modifications are in ${current_config}
Cluster defaults are in ${generic_config}
Check the configuration
cmcheckconf -C ${config}
Apply the configuration
cmapplyconf -C ${config}
Run the new cluster
cmruncl

[Muhammad Ahmad]

hi SASJ,


As 2 nodes cluster is running, any other node with SG (same version installed) and OS (same version installed) is not allowed to run cmquerycl (step #3. in my previous post) in our understanding.

Please Comment.

[AnthonySN]

may we know where you find or read the information that you cannot run a cmquerycl on a non-clustered node.
the error is complaining about permission,pls check the syslog on the non clustered node for errors for more information.