shared volume group

[EAB]

hi,
our O/S hp-ux 11.23
using Service Guard 11.15
oracle RDBMS 9.2.0.4 RAC using RAW Devices
we have 2 instance RAC it was working fine ,
till we shutdown and add new raw devices after that ,, some raw devices exist in one node but doesn't exist in and the other this is an old raw device not a new one
so the 2nd instance can't access the datafile which use this raw device so we close this instance and working only by one
when we chek the vgdisplay at the working node we didn't found the raw device although when we go to the dir for this voloume group and make list we found that ,
ls -ltr
brw-r----- 1 root sys 64 0x060014 Apr 28 2004 fcbrn_brn1
brw-r----- 1 root sys 64 0x060014 Apr 28 2004 rfcbrn_brn1
crw-rw---- 1 oracle dba 64
0x060015 Apr 28 2004 rfcbrn_brn3
crw-rw---- 1 oracle dba 64

The brn1 which is accessible from 2 nodes ,
brn3 accessible only from this node
pls help it is an urgent for our production

[Steven E. Protter]

That is because you can't do that.

You can access a volume group from one node at a time. If you manage to get a second node to access it, it will last about 10 seconds before it melts down.

Here is the way LVM works.

You have a volume group associated with node 1..

On node one, you must run this command:

vgchange -a n

Now on node 2, you run vgchange -a y

Now the volume group is active on node two.

This setup is good enough for Serviceguard high avilability.

If you need both nodes to access data on the volume group at the same time, one of the nodes is going to have to use NFS or CIFS/Samba.

This is a rule of LVM.

SEP

[EAB]

the volume group is already shared ,
and all the raw devices are accesible from 2 nodes except this one. which was working before ,

[EAB]

sorry but i need to add that , the block device for that raw device doesn't exist ,, !!???
i don't know the reason but i think this is the problem ... ?
any advice for this situation...

[harry d brown jr]

Steven, you CAN share a VG between servers using the "shared" option on the VG. The raw devices EAB is talking about are hopefully logical volumes under a shared VG. Anything outside of a VG would probably require Vertias's Cluster FS.

live free or die
harry d brown jr

[EAB]

yes ,
i want to confirm some thing which i found in other forum ,
can i create the block device for this missing raw device by the following

mknod /dev/vg_ops3/fcrbn_brn3 b 64 0x060021

thanks

[Thayanidhi]

Hi,

The raw volume maintenance in RAC nodes are bit tricky.

Follow this (which I follow generally).

From NODE1,

vgchange -a n /dev/vgxx
vgexport /dev/vgxx

From NODE2,
vgchange -a n /dev/vgxx
vgchange -S n -C n /dev/vgxx
vgchange -a y /dev/vgxx

Now do all the LVM operations,

Then fron NODE2,
vgchange -a n
vgchange -S y -C y /dev/vgxx
vgchange -a s /dev/vgxx
vgexport -p -v -m /tmp/mapfile /dev/vgxx

copy the map file to other node then from NODE1,
Create group file under vgxx folder.

vgimport -m /tmp/mapfile /dev/vgxx

vgchange -S y -C y /dev/vgxx (I think it's not required)
vgchange -a s /dev/vgxx

Important note: You have to set the permissions of all the raw device files to the owner:group of oracle. After import the default permission is set for root only. Refer to corresponding oracle documentation.

Regards

TT

[EAB]

we already follow this steps before ,, but there is only missing raw device when we make import for the nod2 , this raw device doesn't exist when we chek at sam at nod1
because of missing block device for this raw device ,
my question is can we create this block device by the following
mknod /dev/vg_ops3/fcbrn_brn3 b 64 0x060021
i test this command at test env. and it is working by i want to confirm that this will not affect the character device
crw-rw---- 64 0x060021 rfcbrn_brn30

and after that we can export from the working nod to the other ,,

pls confirm

[Thayanidhi]

I think you messed the export import process, otherwise lvm device files cannot miss just like that.

I understand now that you are missing a raw volume from both nodes!!

Please confirm, is this your problem?

TT

[EAB]

no ,
the actual situation
at nod1 the raw device fcbrn_brn3 is exist as the following
crw-rw---- 1 oracle dba 64 0x060021 rfcbrn_brn3

but the block device for this raw device doen't exist althoug the other raw devices has 2 file as the following
crw-rw---- 1 root sys 64 0x060021 rfcbrn_brn1
brw-rw---- 1 oracle dba 64 0x060021 fcbrn_brn1
so when i use vgdisplay or sam i didn't find this raw device .. although the oracle instance at this node can use this raw device

so when we used export this raw device doesn't exist in the map file and is not imported to nod2.

any help pls.

[Thayanidhi]

I found from list the one nod1 is having correct permissions for rfcbrn_brn3, but rfcbrn_brn1 has root:sys permissions.
verify that!!

Coming to missing block device file, mknod should work for you. I don't think it will harm you in any case, even if it doesn't work.

Simulate this in a test environment.

Regards

TT

[EAB]

thanks a lot for you reply ,
we will perform nod block device creation after working hours and i'll feed you back.
thanks
Manal

[EAB]

Thanks a lot for all of you it is working after create the block device

Thanks
Manal

[Thayanidhi]

Spare a smal time to analyse the replies and assign points accordingly. It helps the others who is using this forum.