Working with Oracle ASM disks mixed with VxFS

[Ken Englander]

Has anyone come up with a good way (tools, scripts, ?) to distinguish if a LUN is in use by HP-UX before using it with ASM. We are already aware of the information we can see using diskowner and using xd to dump the disk sector 8. We are working with HP-UX 11i v3

It seems that HP-UX will not automatically relaize if a disk is in use by ASM. Also, Oracle will not realize if a disk is in use by HP-UX, even with a mounted filesystem on it.

All feedback is appreciated!

[Steven E. Protter]

Shalom,

soft link the asm disks in a way that will make their use obvious.

That is what I used to do with raw disk areas. Works the same for ASM

SEP

[Ken Englander]

Hi Steven - not sure if I follow - are you suggesting a soft link in /dev? Can you be more specific and possibly suggest how that would help?

Thanks!

[Duncan Edmonstone · Accepted solution]

Ken,

What I *really* *really* hate about ASM is it gets DBAs involved in stuff they shouldn't be involved in - Storage admin is not a DBA task whatever they/oracle think - unfortunately Oracle sees the whole world as an Oracle database...

I wouldn't let ASM anywhere near the contents of /dev/rdsk or /dev/rdisk... If I want a disk to be used for ASM, then I'll identify its major and minor number using

ll /dev/rdisk/disk10

or whatever, and then create a new DSF specifically for Oracle ASM using mknod with the major/minor numbers I identiified:

mkdir /dev/asm
mknod /dev/asm/asmdisk1 c 23 0x000005
chown oracle:dba /dev/asm/asmdisk1
chmod 640 /dev/asm/asmdisk1

or whatever...

Then tell the DBAs to set the Oracle ASM parameter ASM_DISKSTRING to /dev/asm

This keeps ASM away from my disks, and has the added value of not resetting the ownership on the disk to bin:sys or root:sys every time insf is run...

HTH

Duncan

[Turgay Cavdar]

I think the best way to recognize ASM disk is using:
#od -c /dev/rdsk/c5t1d2| head -10
and check for "ORCLDISK" pattern.

Actually pvcreate command recognizes the ASM disk, check the man page of pvcreate.:
"Furthermore, when the -f option is not specified, the operation is denied if pv_path already belongs to another LVM volume group, or the pv_path refers to a disk device under the control of the VERITAS or Oracle ASM Volume Manager."

For easy administration you can create seperate directory for device files for ASM, and give easy names to them.
#mkdir /dev/oracle
#mknod /dev/oracle/raw_dbname_asm1_120g c 22 0x00001e (major/minor numbers belong to original disk)

[Zinky]

Ken, Shalom:

In my shop, I use a Volume Manager between ASM and my OS. I use VxVM as that "iterim" layer. That keeps tracking hugely easier.

There rules however. Carve out volumes using whole LUNs - do not stripe of concat.

for example:

/dev/vx/rdsk/asm_sapdg/asvmvol01

where asmvol01 is contained on a single EVA, XP, EMC or whatever disk.

Under Oracle -- the DBA will simply have their disksstring set to:

/dev/vx/rdsk/asm_sapdg/asm*

I beleieve the same can be done with LVM even as your volume manager.

I know there'll be others who will say -- the Volume Manager layer is an unnecessary overhead. So far though we're not "feeling" it and is GOOD to have the peace back in our ecosystems.

Since you obviously will have a tall time convincing to introduce a volume manager layer between HP-UX and Oracle, I suggest then you write a script (there should lots of examples out there) that will summarize what disks belong to what ASM diskgroup... it should be relatively easy.

[Kevin Gaffney]

I've been working on this very same issue for 6 months. The only thing I've found was you can change the ownership to oracle:dba and use diskowner before adding storage to anything. Duncan Edmonstone is right and wrong. If you use the "insf" with the "-e" option it will reset back to bin:sys. As long as you stay away from the "Reinstall (-e)" option you should be ok.

[Ken Englander]

You all make good points, many of which we have discovered - I just never got back to this thread.

We did find a gotcha when setting up separate DSFs for ASM. Ignite does not support them, so they do not come back the way you want. It is something you can script around, but that is just another thing to do. Sorry, but I cannot recall the details of what it does.

Thank you all for your input. Sorry I did not assign points a LONG time ago.