creating a volume group when using clusters

[John Jimenez]

Hey guys,
Soon we will be installing a cluster (Oracle RAC), I am new to the clustering. The RAC will be installed next week, but I was told to create the volume groups on both servers. But hosts 2 knows that the volume group already belongs to the volume group on host1. Does the RAC have to be built first?
Any input to this new clustering world I am now in, will be greatly appreciated.

host2:[/]
# pvcreate /dev/rdsk/c9t0d1
pvcreate: The physical volume already belongs to a volume group
# mknod /dev/vg02/group c 64 0x020000
host2:[/]
# vgcreate -e 10000 -s 4 /dev/vg02 /dev/dsk/c9t0d1
vgcreate: Volume group "/dev/vg02" could not be created:
Device busy
host2:[/]

[Jeff_Traigle]

The disks are seen by the systems in a cluster like this. You create once on one system, create a map file of the configuration, copy the map to the other nodes, and import the configuration there.

On node 1:

vgexport -p -s -m vg02.map vg02

On node 2:

mkdir /dev/vg02
mknod /dev/vg02/group c 64 0x020000 (check the original node "ls -l" for this hex number and make it match)
ftp, scp or whatever the vg02.map from node 1
vgimport -s -m vg02.map

The Serviceguard documentation, like all HP documentation, is at docs.hp.com. Would be better to get some training before delving into a production environment, but I know how some companies are about offering training.

[Jeff_Traigle]

Oops... that vgimport command should be:

vgimport -s -m vg02.map vg02

[Simon Wickham_6]

Hi,

Issue a vgchange to make it cluster-aware:
$ vgchange -c y

Regards,
Simon

[John Jimenez]

Thanks for the enlightenment. feel free to add on any other useful info. I will leave this thread open a couple of more days.

[John Jimenez]

I have one more question in regards to clustering using Oracle RAC (I have before never used Oracle either).

Lets say I created the minor number 0x020000 for vg02 on host1. When I create the minor number in hosts2. Do I need to use a differnet minor number? or can I use 0x020000 again?

I have some Oracle documentation. When creating the volume group on host1, it mentions not to use the same minor number on the same cluster. But when it talks about the mknod on host2 it does not mention this point again.

I would think that the directions would mention this point when creating vg02 on host2, and that maybe it is a misprint and should have said not to create the same minor number on the host, instead of on the cluster.
Does it matter one way or the other?

[John Jimenez]

Maybe I should open a new thread

[Jeff_Traigle · Accepted solution]

You don't want the same minor number for different volume groups. In fact, I think HP-UX gives an error if you attempt to do so. The minor numbers for the same volume group on the different nodes needs to be the same, as I recall, otherwise the vgimport will fail. (Someone can correct me if I'm wrong, but I seem to recall it works that way.)

[Sundar_7]

Though the minor numbers for the same volume group in the cluster nodes need not be the same, it is a good practice to keep it the same.

Nope, the vgimport will NOT fail if the minor number is different.

I know in one case where the minor number MUST be kept the same between the cluster nodes. That is when the volume group is hosting the NFS filesystems. My understanding is the NFS file lock also includes the minor number of the VG.

[Deoncia Grayson_1]

As a general rule of thumb you want a shared volume group to have the same minor number on both nodes, but you cannot have the same minor number on the same server, this will cause an error.

for ex:

vg03 on servera
0x040000

vg03 on serverb
0x040000

[Jeff_Traigle]

That must be where I was remembering it from... the NFS module of the Serviceguard class. Whether required or not, it does make your administrative life much easier if you keep as much as possible symmetrical between the nodes as possible.

[TwoProc]

John,

I've found life a lot easier if I make sure that the group numbers are unique across all servers in the cluster. And, I do it for ALL vg's in all of the clustered servers. True enough, you don't HAVE TO. But, you'll certainly avoid any chance of problems later on. I do not apply that rule for root (vg00) on both servers, but you can extend that to any vg's comprised of local disks that won't be moving with the package.

At a minimum, the vg's group number needs to be unique for all clustered vg's in the cluster.

So, if you've got disks that are moving between machines, then those numbers should be unique between all the machines, and not collide with ANY vg's group numbers (local or not) on all of the machines in the cluster.

Which I just simplify to the sysadmin's in our group by saying "all vg's group numbers are unique across the WHOLE SAN" - which is an over-the-top way to avoid any problems down the line, but I do it anyway.

[John Jimenez]

Thanks guys. Yes I would prefer to keep it the same too. Thanks for clearing that up for me.