Matching dsk device files on two servers to one EMC

[Philip Kernohan]

We have two servers running UX-11.0 connected to an EMC array hosting an Oracle8i database.

We have configured the hardware on each of the servers such that each hardware address 7/0/0, 8/0/0 etc. have corresponding fiber channels. The EMC has been configured to present the same devices on pairs of FCs to each server. Each server sees the correct number of devices on each FC and they are the same devices.

One server currenty has the devices mapped into VGs and LVs and is running Oracle.

Problem: we would like to be able to turn down one server and turn up the other and still have the Oracle database intact. To do this we assumed if the devices were named the same then the VGs and LVs could be restored to either server.

The hardware paths are different: 7/0/0.8.0.18.2.9.1 on one server presents itself as 7/0/0.8.0.20.2.9.1 on the other.

The device files are named differently: c94t9d1 on one server presents itself as c34t9d1.

Is it possible to re-create the VGs and LVs on one server exactly as they were on the other and hence make the database function?

Many many thanks in advance,
Philip

[Ken Scharpell]

TO match your EMC disks use the EMC inq or syminq command (somewhere under /usr/emc or /usr/symcli/bin). This does basically an EMC version of ioscan. However, it lists the unique serial number of each disk, eg;

Under the SER NUM column the unique id is digits 3-5. eg syminq lists 63041311, the unique serial of that disk is 041 which you can see from the same command on the other server.

One you id the disks and their device files then YES, you can create a VG, and use vgimport to import the lvols on it from the devices you list. Or just use vgscan to identify them.

[Joel Shank · Accepted solution]

Hi Philip

Yes, you can do this. In fact, this is what Service Guard does. Are you running this under Service Guard? In either case, this is what you have to do:

Assume you have the VG and LVs all defined properly on NODE1 and you know all the disk device names that are assigned to the VG on NODE1 and there corresponding device names on NODE2.

1. Create a map of the VG on NODE1:
vgexport -p -m /tmp/vg.map vgname

2. Copy the map to NODE2:
rcp /tmp/vg.map NODE2:/tmp/vg.map

3. On NODE2, create the VG definition and group file:
(1) mkdir /dev/vg
(2) mknod /dev/vg/group c 64 0x??0000
(Replace ?? with a valid number minor number for NODE2)

4. On NODE2, import the volume group:
vgimport -m /tmp/vg.map vgname List-Of-Volumes
(The List-Of_Volumes are the volumes that NODE2 sees that correspond to the volumes on NODE1 that are assigned to the volume group)

The volume group is now defined on NODE2 exactly as in NODE1.

BE CAREFUL however. If you are not usinf Service Guard (or some other software that will coordinate the volume group across the two servers, it is possible to have the volume group activated on both servers at the same time, and this will corrupt the volume group. You can use /etc/lvmrc to insure the volume group is not automatically activated when the server reboots, which is a good idea. Edit /etc/lvmrc and set the variable AUTO_VG_ACTIVATE to 0, then go down in and add
parallel_vg_sync /dev/vg00 in the custom_vg_activation function so that vg00 will be activated at reboot. Other volume groups can be added also, so they will be actiavted at reboot.

Hope this helps. Good Luck?
JLS

[Rita C Workman]

I agree with Joel.
What you are asking for is something to cover your system for failover. That would be MC/ServiceGuard. You can create a clustered environment whereby nodes could failover to what/where you want them to and control what packages are permitted to run in these events. And with MC/SG it will maintain the integrity of what app owns what devices....(and data..)

[Philip Kernohan]

Many thanks to Ken, Joel and Rita.

Joel, I believe you have nailed this one and I shall be trying it out this evening. I'll post ONE more message in appreciation of my findings.

Thanks again,
Philip

[Philip Kernohan]

Finally ... Joel I tried your solution out and it works a treat for a mountable Unix vxfs filesystem and am assuming raw filesystems will be handled the same.

With the VGs set up on both servers and a few 'vgchange -a y/n ' commands I can control which server holds the VG.

Thank you all for your pointers on MC/SG - we'll look into that also.