Oracle 10g Release 2 RAC with ASM on HP-UX 11.11

[Wachyu Danyanto]

Hi,

I have a problem during creation of ASM diskgroup in Cluster Environment (Oracle 10g Release 2) on HP-UX 11.11.

I have some questions related to Oracle 10g R2 RAC with ASM on HP-UX 11.11:
1. If ASM going to used for storage of Oracle Database, does it need ServiceGuard to be activated? If it doesn't need ServiceGuard, why I cannot create ASM Diskgroup pointing to the raw disk (/dev/rdsk/c6t0d0) ?

The errors say:
ORA-15018: diskgroup cannot be created
ORA-15030: diskgroup name "DGSYSTEM" is in use by another diskgroup

2. I have another error when try to running srvctl:

$ srvctl config
PRKH-1010 : Unable to communicate with CRS services.
[PRKH-1000 : Unable to load the SRVM HAS shared library
[PRKN-1008 : Unable to load the shared library "srvmhas10"
or a dependent library, from
SHLIB_PATH="/apps/oracle/product/10.2/crs/jdk/jre/lib/PA_RISC2.0:/apps/oracle/product/10.2/crs/jdk/jre/lib/PA_RISC2.0/server:/apps/oracle/product/10.2/crs/jdk/jre/../lib/PA_RISC2.0:/apps/oracle/product/10.2/crs/lib32:/apps/oracle/product/10.2/crs/srvm/lib32:/apps/oracle/product/10.2/db/lib32:/apps/oracle/product/10.2/db/rdbms/lib32:/usr/lib"
[java.lang.UnsatisfiedLinkError: /apps/oracle/product/10.2/crs/lib32/libsrvmhas10.sl: specified library, or one of its dependencies, does not exist.]]]


Could anyone help me with this?
Does anyone have the same experiences issues like this?

Thanks in advance.

Regards,
Danyanto

[Wachyu Danyanto]

Hello,

Is this forum active?

Regards,
Danyanto

[Zinky]

Yes we're alive.

The non-response can only mean one thing -- possibly no one's using ASM in this forum.

I myself have doubts using ASM unless the storage, sysadmin and dba groups are part of a homogenous organization.

Say -- what kind of array are you using with your ASM implementation?

[Wachyu Danyanto]

"
Yes we're alive.

The non-response can only mean one thing -- possibly no one's using ASM in this forum.

I myself have doubts using ASM unless the storage, sysadmin and dba groups are part of a homogenous organization.

Say -- what kind of array are you using with your ASM implementation? "

Hi,

I believe in HP/Oracle CTC, there are someone have tried Oracle 10g R2 RAC with ASM on HP-UX 11.23.
Because they have write cookbook about installation of Oracle 10g R2 on HP-UX 11.23.

About the storage, we are using HP Storage 12000 (OPENV).

About the operating system, we used HP-UX 11.11.

Is there any problem with ASM without ServiceGuard in HP-UX 11.11?

Thank you.

Regards,
Danyanto

[Zinky]

You will be better off running RAC on HP-UX 11i v2 (11.23). Veritas components on HP-UX 11.11 are pretty dated (based on VxVM 3.5/VxFS 3.5).

Also, you have the perfect array - XP12000 for your cluster/RAC/ASM implementation. ASM considers Hitachi based storage frames like the XP12000 as one of its fully certified storage arrays.

If you are testing/playing Oracle RAC with ASM implementation -- PLEASE read through the requirements/manuals very thoroughly.


Selamat Tembang.

[Zinky]

Please see also:

http://www2.itrc.hp.com/service/james/dispDoc.do?docURL=http%3A%2F%2Fsearch.hp.com%2Fredirect.html%3Furl%3Dhttp%253A%2F%2Fdocs.hp.com%2Fen%2F8831%2F10gRAC%26qt%3D%252BXP12000%26hit%3D19&aid=SEARCH_MANUAL&pil=17&serStr=XP12000

[Duncan Edmonstone]

Generally you don't use ASM with Serviceguard - if you want to use ASM you use a complete Oracle stack - no HP clustering products at all *unless* you want to use SLVM - in that case you can use ASM with Serviceguard.

As to the other messages - I'm no RAC DBA, but I would certainly fix your CRS problem before attempting *anything* else. IIRC ASM relieas on CRS so you need to fix that problem before touching ASM. A quick google on the problem seems to indicate this can arise when libclntsh shared lib isn't created in CRS_HOME. Do you have a libclntsh.sl file in CRS_HOME/network/lib ?

The ASM problem may just be a symptom on the CRS problem, or it could be that ASM is finding existing ASM headres on the disk in question. If thats the case and you're *completely sure* the disk isn't in use by any system, then just writing over the start of the disk should let ASM use it:

dd if=/dev/zero of=/dev/rdsk/c6t0d0 bs=8192 count=1000

HTH

Duncan

[Wachyu Danyanto]

Hi Nelson,

Thank you for your reply.
About the technical reference Guide about Oracle 10g RAC using SGeRAC. But unforunately we are not using ServiceGuard for the ASM. Only raw device. ServiceGuard packages is already installed but not configured yet.

About the operating system, Oracle said it compatible with the HP-UX 11i v1.
Is it possible not to upgrade the Operating System to 11.23?

Thank you.

Regards,
Danyanto

[Duncan Edmonstone]

So you are using Oracle RAC 10g with Oracle's own clusterware, but you are *also* going to use Serviceguard on this system as well?

I think you need to get this checked with both youre Oracle and HP support people, I doubt either would be happy with two seperate cluster managers running on the same nodes - who is responsible for fencing?


HTH

Duncan

[Wachyu Danyanto]

"
Generally you don't use ASM with Serviceguard - if you want to use ASM you use a complete Oracle stack - no HP clustering products at all *unless* you want to use SLVM - in that case you can use ASM with Serviceguard.

As to the other messages - I'm no RAC DBA, but I would certainly fix your CRS problem before attempting *anything* else. IIRC ASM relieas on CRS so you need to fix that problem before touching ASM. A quick google on the problem seems to indicate this can arise when libclntsh shared lib isn't created in CRS_HOME. Do you have a libclntsh.sl file in CRS_HOME/network/lib ?

The ASM problem may just be a symptom on the CRS problem, or it could be that ASM is finding existing ASM headres on the disk in question. If thats the case and you're *completely sure* the disk isn't in use by any system, then just writing over the start of the disk should let ASM use it:

dd if=/dev/zero of=/dev/rdsk/c6t0d0 bs=8192 count=1000

HTH

Duncan "

Hi Duncan,

Yes, we are planned that we don't use ServiceGuard for the ASM, only raw disk devices.

About the libclntsh.sl , yes it is there.
CRS is running well, but after I patched the clusterware to 10.2.0.3, it seems that it have a problem with the srvctl utility.

Before I created diskgroup in the ASM, I have dd all the raw disk devices. All of it from /dev/rdsk/c6t0d0 - /dev/rdsk/c6t0d3.
But when creating Diskgroup it showing error
like this:

create diskgroup dgdata02 external redundancy disk '/dev/rdsk/c6t0d2'
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15030: diskgroup name "DGDATA02" is in use by another diskgroup

I have tried with all of the raw disk, it all show the same error.

This ASM problem occur first time before I patch to 10.2.0.3. I read from Oracle Metalink article about this ASM error and I was think this is bug from Oracle 10.2.0.1, so I patched the Oracle clusterware and database software to 10.2.0.3. But after been patched, the Error on ASM still don't solve yet, still showing the same error and also another error is come when I want to use srvctl utility.

I have repeatedly installed Oracle clusterware and Database many many times, but the ASM Problem didn't solve yet.
The Production server needs to go alive soon after ASM is created then I need to configuring RAC Database on ASM and then the production database server will be ready for used.

Really need help on this.
Thank you very much.

Regards,
Danyanto

[Wachyu Danyanto]

Hi Duncan,

We are planned to used ASM on raw devices only not ASM on SLVM so we are not using ServiceGuard. The ServiceGuard packages already installed because the ServiceGuard packages already preinstalled at the both of server and we also have the license for the ServiceGuard.
The ServiceGuard have not been configured yet nor activated yet, it just have the packages installed but the cluster manager is not running.

The Installation of Oracle 10g R2 Clusterware and Database Software already finish successfully, but have some problem with the ASM when creating Diskgroup on Raw Disk Device. If this error doesn't solve yet until this week, We have planned to use ServiceGuard and configuring Raw Logical Volume for the ASM Instance.

Is it the error in ASM related to the ServiceGuard packages that have been installed on the system?

Do I have to remove it from the system and see whether the ASM will running well?

Thank you.

Regards,
Danyanto

[Zinky]

Wachyu,

Long shot -- but if you're using ASM on Raw and references to "diskgroup" - then I suggest you look at your environment specifically the following areas:

Are you licensed for Full VxVM and CVM (Cluster Volume Manager)?

Can you post: "vxlicrep" output?

[Duncan Edmonstone]

Danyanto,

No, I doubt Serviceguard beibg installed has anything to dow ith your ASM problems - what I am saying is that if you intend to use Serviceguard on these systems in the future as well as Oracle RAC with ASM, then you are effectively running two seperate cluster products on the same nodes - I don't think either Oracle or HP are going to like that - never mind what your current problems are, if both HP and Oracle won't support your configuration you are gonna be in trouble.

With respect to your srvctl problem. I would use the chatr command to check out all shared libraries specified in:

-the srvctl binary
-the libsrvmhas10.sl shared library.

e.g. here's the output from chatr on my box:

chatr ./app/oracle/product/10.2.0/lib/libsrvmhas10.sl
./app/oracle/product/10.2.0/lib/libsrvmhas10.sl:
64-bit ELF shared library
shared library dynamic path search:
LD_LIBRARY_PATH enabled first
SHLIB_PATH enabled second
embedded path enabled third /oracle/app/oracle/product/10.2.0/lib/
shared library list:
libhasgen10.sl
libocr10.sl
libocrb10.sl
libocrutl10.sl
libclntsh.sl.10.1
libskgxn2.sl
libttsh10.sl
libcl.2
librt.2
libpthread.1
libnss_dns.1
libdl.1
libnsl.1
libm.2
libons.sl
libdbcfg10.sl
shared library mapped private disabled
shared vtable support disabled
explicit unloading enabled
linkage table protection disabled
segments:
index type address flags size
5 text 4000000000000000 z---c- D (default)
6 data 8000000100000000 ---m-- D (default)
static branch prediction disabled
kernel assisted branch prediction enabled
lazy swap allocation for dynamic segments disabled
nulptr dereferences trap disabled

make sure all those libraries mentioned are all in directories in your SHLIB_PATH somewhere.

As I said before - I wouldn't do anything with ASM until your are completely satisfied that crs and utilisties such as srvctl are behaving as they should.

HTH

DUncan

[Wachyu Danyanto]

Hi Nelson,

We don't use Veritas for the CVM and we don't have license for it.

Here is below the output of vxlicrep:

VERITAS License Manager vxlicrep utility version 3.00.007
Copyright (C) VERITAS Software Corp 2002. All Rights reserved.

Creating a report on all VERITAS products installed on this system

Error: There are no valid VERITAS License keys installed in the system.

Regards,
Danyanto

[Wachyu Danyanto]

Hi Duncan,

Below is the output of chatr:

$ chatr /apps/oracle/product/10.2/db/lib/libsrvmhas10.sl
/apps/oracle/product/10.2/db/lib/libsrvmhas10.sl:
64-bit ELF shared library
shared library dynamic path search:
LD_LIBRARY_PATH enabled first
SHLIB_PATH enabled second
embedded path enabled third /apps/oracle/product/10.2/db/lib/
shared library list:
libhasgen10.sl
libocr10.sl
libocrb10.sl
libocrutl10.sl
libclntsh.sl.10.1
libskgxn2.sl
libttsh10.sl
libcl.2
librt.2
libpthread.1
libnss_dns.1
libdl.1
libnsl.1
libm.2
libons.sl
libdbcfg10.sl
shared library mapped private disabled
shared vtable support disabled
explicit unloading enabled
linkage table protection disabled
segments:
index type address flags size
5 text 4000000000000000 z---c- D (default)
6 data 8000000100000000 ---m-- D (default)
static branch prediction disabled
kernel assisted branch prediction enabled
lazy swap allocation for dynamic segments disabled
nulptr references disabled


Regards,
Danyanto

[Duncan Edmonstone]

So have you checked that *all* these libraries are somewhere in a directory on your SHLIB_PATH ?

libhasgen10.sl
libocr10.sl
libocrb10.sl
libocrutl10.sl
libclntsh.sl.10.1
libskgxn2.sl
libttsh10.sl
libcl.2
librt.2
libpthread.1
libnss_dns.1
libdl.1
libnsl.1
libm.2
libons.sl
libdbcfg10.sl


HTH

Duncan

[Wachyu Danyanto]

Hi Duncan,

It seems that not all libraries exists on the lib directory of Oracle.

$ ls -l libcl.2
libcl.2 not found
$ ls -l librt.2
librt.2 not found
$ ls -l libpthread.1
libpthread.1 not found
$ ls -l libnss_dns.1
libnss_dns.1 not found
$ ls -l libdl.1
libdl.1 not found
$ ls -l libnsl.1
libnsl.1 not found
$ ls -l libm.2
libm.2 not found
$ ls -l libons.sl

Where are they gone?
And what should I do about the missing library?

Thank You.

Regards,
Danyanto

[Duncan Edmonstone]

Are those files not in /usr/lib/pa20_64 or /usr/ccs/lib/pa20_64 - they are searched by default when the dlopen call is made for 64-bit binaries (dlopen is what loads a shared library 'man dlopen' for more details)


HTH

Duncan

[Wachyu Danyanto]

Hi,

Those files are in /usr/lib/pa20_64.


Regards,
Danyanto

[Duncan Edmonstone]

Running out of ideas now..

Have you checked that none of those libraries are symbolic links to files that don't exist?

If thats not it then you might want to install tusc:

http://hpux.connect.org.uk/hppd/hpux/Sysadmin/tusc-7.8/

and then run your command with that - might give some pointers to your problem:

tusc srvctl config

HTH

Duncan

[Duncan Edmonstone]

Something else to try:

$ export SRVM_TRACE=1
$ srvctl config


Should give you some more debug info - maybe that will help.


HTH

Duncan

[Duncan Edmonstone]

And something else... I've been mistakenly getting you to look at your 64-bit libsrvmhas10.sl file. Reading through your posts agin, I realise we should be looking at the 32-bit one.

Do what you did before for this sl:

chatr /apps/oracle/product/10.2/crs/lib32/libsrvmhas10.sl

HTH

[Zinky]

Wachyu, et al.

I thought on HP-UX that whenever there are references to CVM and Diskgroup -- it means VxVM as the volume manager.


I have not yet read through manuals, docs, etc. about what RAC implementation really entails but the mebntion of CVM and "Diskgroup" really pointed me to the direction that perhaps if you're using CVM/ASM with RAW -- that you need VxVM?

I am also in the R&D phase of the possibility in getting our environment into an HA environment. I am not a DBA - I am more of a UNIX/Storage Admin.

Let us know.

[Duncan Edmonstone]

Nelson,

diskgroup is also an ASM term.

ASM is Oracle's pseudo volume manager, which does away with the requirement for any other volume manager or filesystem

If you choose to implement RAC with a complete Oracle stack (i.e. no HP Serviceguard or Veritas Cluster Server), then you use ASM to provide cluster-wide storage management.

If you want to know more, you can read up on ASM here:

http://www.oracle.com/technology/products/database/asm/index.html

Me, I still trust LVM/SLVM/VxVM/CVM more than I do ASM

HTH

Duncan

[Zinky]

Mucho Graci Duncan!

A DBA friend mentioned ASM uses a subset of VxVM in its ASM as well as ASL (array suport library) pieces from Veritas/Symantec - any truth/hints to this?

Oracle as a long history of using other technologies and bundling them -- like Legato Networker was briefly integrated with Oracle...