Oracle 10g Release 2 RAC with ASM on HP-UX 11.11

[Duncan Edmonstone]

Something else to try:

$ export SRVM_TRACE=1
$ srvctl config


Should give you some more debug info - maybe that will help.


HTH

Duncan

[Duncan Edmonstone]

And something else... I've been mistakenly getting you to look at your 64-bit libsrvmhas10.sl file. Reading through your posts agin, I realise we should be looking at the 32-bit one.

Do what you did before for this sl:

chatr /apps/oracle/product/10.2/crs/lib32/libsrvmhas10.sl

HTH

[Zinky]

Wachyu, et al.

I thought on HP-UX that whenever there are references to CVM and Diskgroup -- it means VxVM as the volume manager.


I have not yet read through manuals, docs, etc. about what RAC implementation really entails but the mebntion of CVM and "Diskgroup" really pointed me to the direction that perhaps if you're using CVM/ASM with RAW -- that you need VxVM?

I am also in the R&D phase of the possibility in getting our environment into an HA environment. I am not a DBA - I am more of a UNIX/Storage Admin.

Let us know.

[Duncan Edmonstone]

Nelson,

diskgroup is also an ASM term.

ASM is Oracle's pseudo volume manager, which does away with the requirement for any other volume manager or filesystem

If you choose to implement RAC with a complete Oracle stack (i.e. no HP Serviceguard or Veritas Cluster Server), then you use ASM to provide cluster-wide storage management.

If you want to know more, you can read up on ASM here:

http://www.oracle.com/technology/products/database/asm/index.html

Me, I still trust LVM/SLVM/VxVM/CVM more than I do ASM

HTH

Duncan

[Zinky]

Mucho Graci Duncan!

A DBA friend mentioned ASM uses a subset of VxVM in its ASM as well as ASL (array suport library) pieces from Veritas/Symantec - any truth/hints to this?

Oracle as a long history of using other technologies and bundling them -- like Legato Networker was briefly integrated with Oracle...

[Duncan Edmonstone]

> A DBA friend mentioned ASM uses a subset of VxVM in its ASM as well as ASL (array suport library) pieces from Veritas/Symantec - any truth/hints to this?


Not that I'm aware of. I doubt this would be the case as ASM is a serious competitor to Seritas/Symantec on UNIX systems now, and Oracle give it away with the database.

ASM doesn't have any kind of array support librarys (in fact you need to implement MPIO yourself in some way, ASM won't do that for you)

HTH

Duncan

[Wachyu Danyanto]

Hi all,

I have just scratch the installation again and reinstalling again the oracle 10g Release 2 Clusterware and Database software.

After the installation finished I try to creating ASM instance again and creating diskgroup. It keep show the error :
ORA-15018: diskgroup cannot be created
ORA-15030: diskgroup name "DGSYSTEM" is in use by another diskgroup

I try to find out why the ASM Diskgroup still cannot be created. And finally I know there is a problem with ASM if the raw disk device (LUN) is having multipath or alternatif link path (redundant HBA).

The problem is in Oracle itself. This is very surprising me.

Example for the /dev/rdsk/c6t0d0 is having the alternatif link path is /dev/rdsk/c8t0d0.

So the when creating ASM Diskgroup, it will detect that 2 LUN (raw disk path), so it always say "diskgroup is in use by another diskgroup".

About the other error that showing when using srvctl, it seems another oracle bugs when the Oracle 10g Release 2 (10.2.0.1) have been patched to 10.2.0.3, because when I install the Oracle 10g Release 2 base software (10.2.0.1), the srvctl utility is running well and fine.

So this problem is solved, wow right now I am feeling better because the production can go live soon.

Really thank you for all of your kind answer, brainstorming, and all the support.

I have another one question,
How to configure default stack size (kbytes) in HP-UX?
If we configure in the Kernel (maxssize), it doesn't change the default value that show when running 'ulimit -a'.

$ ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 1048576
stack(kbytes) 131072
memory(kbytes) unlimited
coredump(blocks) 4194303
nofiles(descriptors) 60


Thank you very much all.


Regards,
Danyanto

[Zinky]

Hmm, then Oracle ASM (CVM) always expects the disk device it sees to be already protected? And if it sees the subpath instances of the same LUN/Disk it will croak?

Wow.. this could be an issue or something is not correct with your installation.

Apart from SecurePath for Active-Passive (for older EVAs), there is no Multipathing software that "masks" the other paths of a multi-pathed disk. Each cXtYdZ instance of a LUN/Disk will always be seen by the OS and it will be up to the volume manager to figure out which is the active/primary path (i.e. PVLINKS or DMP or MPIO)...

Please let us know how your adventure goes...

[Duncan Edmonstone · Accepted solution]

ASM has a special initialization parameter called asm_diskstring which allows you to limite what ASM can see, so for instance if your were using HP SecurePath for A/A disk arrays (or EMC Powerpath or Hitachi HDLM) and you had 2 paths to your disk:

/dev/rdsk/c1t0d0
/dev/rdsk/c2t0d0

You can set asm_disk_syting as follows:

asm_disk_string=/dev/rdsk/c1*

And Oracle will only see one path to the disks.

I actually prefer to use mknod to create a whole seperate set of device files for Oracle (matching major and minor numbers from the real disks). This allows me to get round the problem of ownership (after every reboot or run of 'insf -e' all those device files you carefully chown'd to oracle are owned by root again) and puts all oracle disks in a unique directory (such as /dev/asm)

You can also run ASM over VxVM/CVM if you wish and use DMP instead.

HTH

Duncan

[Wachyu Danyanto]

Hi Nelson,

There is not something wrong with the installation, because the installation is successfully, this is just Oracle bugs in ASM.
If ASM sees multipath raw disk devices, it will failed to mount.

Hi Duncan,

Your answer is correct about the asm_diskstring parameter.
I used that parameter to define only one path to the raw disk and finally the ASM Diskgroup can be created.

And yes, about the insf -e, would be better if we created a special device, because when you run insf -e it will revert back the original disk owner to root.

Thank you.

Regards,
Danyanto