CA Failover testing

[Kehad Snydewel]

Hi all.

I have 2 EVA 6000s. One at the Production Site (Source Storage System) and one at the DR Site (Destination Storage System) with CA Replication between the two Sites.

2 hosts: Host 1 - SAPPRD1 which is the Production Server and Storage is presented to the host from the Production EVA (source);
Host 2 - SAPPRD2 which is the DR Server and Storage is presented to the host from the DR EVA (destination). SAPPRD2 is the backup server of SAPPRD1

SAPPRD1 Vdisks is being replicated to the DR EVA. I want to test the CA replication by failing over the Source EVA DR group to the destination EVA and present the replicated LUNs to SAPPRD2.

Please check and comment on my action plan regarding the CA failover procedure:

Action plan:

1. Check the state of the DR Group. If logging, merging, or full copy is occurring, wait for completion.
2. Stop host (SAPPRD1) I/O to the Vdisks that will be failed over and shutdown host to flush SAP Database.
3. Fail over DR group to the destination EVA.
4. Present the Vdisk to the OS (HPUX11.11) of the host at the net destination site.

My SAP skills are quite scary so the DBA will do some changes on SAP and I only on the SAN and UNIX hosts.

Question:
On SAPPRD1 I have volume group VGSAP and on SAPPRD2 as well. I have Vdisks presented to SAPPRD2. Should I unpresent the Vdisks from SAPPRD2 before presenting the replicated Vdisks to the host?

Please see sloppy drawing attached.

You time and advice is highly appreciated!!

[Ivan Ferreira · Accepted solution]

1- What operating system are you using?1. Check the state of the DR Group. If logging, merging, or full copy is occurring, wait for completion.

Correct. Ensure to clic the "refresh" button before doing anything.

2. Stop host (SAPPRD1) I/O to the Vdisks that will be failed over and shutdown host to flush SAP Database.

Correct, ensure to umount the file systems.

3. Fail over DR group to the destination EVA.

Ok. Do not perform a "failback" before 15 minutes.

4. Present the Vdisk to the OS (HPUX11.11) of the host at the net destination site.

Correct.

You will have to perform an ioscan and import the volume groups with their respective devices.

[Kehad Snydewel]

The OS on both servers are HPUX11.11.

What should I do with the Vdisks currently presented to SAPPRD2 before presenting the Replicated Vdisks?

I will export VGSAP on host SAPPRD1 and Import VGSAP on SAPPRD2.

[Ivan Ferreira]

Unless the mount point is the same, you could leave it as it is. Even when the volume groups names are the same, the volume group ID should be different, and you can import it with another name.

You don't need to remove the presentations from the current disks.

[Kehad Snydewel]

SAPPRD1 is a Replica of SAPPRD2 so the LV in VGSAP and mount points will be the same but outdated because a Database refresh has not been done for about 6 months.

So can I deactivate VGSAP on SAPPRD2 and import VGSAP of SAPPRD1 as VGTEST for example?

[Ivan Ferreira]

Yes, that can be done:

1. make the volume group unavailable

vgchange -a n /dev/vgsap

2. Export the the disk while creating a logical volume map file.

vgexport -v -m data_map vgsap

4. Move the data_map file to the new system.

5. On the new system recreate the volume group directory

mkdir /dev/vgtest
mknod /dev/vgtest/group c 64 0xNN000

6. Identify the disks with ioscan in the other system.

7. Import the disks to the new system

vgimport -v -m data_map /dev/vgtest /dev/dsk/c2t1d0 /dev/dsk/c2t2d0

8. Enable the new volume group

vgchange -a y /dev/vgtest

[Kehad Snydewel]

Hi Ivan,

The failover of the Replicated vdisks and presentation to SAPPRD2 was successful.

We imported and mounted the files systems but could not start the listener. We received the following error:

Message 1070 not found; No message for product=network, facility=TNSTNS-125
45: Message 12545 not found: No message file product=network, facility=TNS
TNS-12560: Message 12560 not found; no message file for product=network, facility-TNS
TNS-00515: Message 515 not found; No message file for product=network, facility=TNS
HPUX Error: 2: No such file or directory

Sapprd2 23:

We failed back and booted SAPPRD1 but the host did not boot and dropped into a shell. I booted into single user mode and removed all VGSAP entries in the /etc/fstab file and the server booted normally. I did an ioscan and autopath display and all the vdisks was intact as before on SAPPRD1. I replaced the /etc/fstab file and mounted the vgsap files systems but 2 LV could not be mounted.

vgdisplay â v vgsap showed that 2 LVs had 0KB. I deactivated vgsap and when I activate vgsap it returned with an error stating â there are stale partitions on some or more Logical Volumesâ .

I had to restore the 2 LVs from D.P backups.

Do you perhaps know way this happened?

[McCready]

Well, I can't immediately answer why the above happened, but I can share how we do CA testing so we don't worry about the failback corruption issues...

Assuming you have the extra disk space, you can make a snapclone of each the replicated disks (while suspending replication for the entire disk group to get a consistent copy) into containers (pre allocated disk space that matches the size of the replicated disks)to reduce the overall time for the snapclones to complete. Once all snapclones are started, we then resume replication so we minimize the time we don't replicate.

One then presents the snapclones to the DR systems, and you don't have to worry about the original disks, as they were never touched by any other system. Of course, you would eventually want to figure out what the problem was, if nothing else to prove you could fail back to the original site (in our case, starting replication of the snapcloned copies back to the original location).

This method allows you, depending on application design, to do your testing during regular work hours or with a snap that was take anytime during the "application" day or month.