Migrate data from one disk array to another

[HPUX SysAdm]

What is the best way to do the above migration keeping in mind the following:

- production environment
- minimize downtime
- fast and efficient

The server is part of a two-node ServiceGuard cluster running HP-UX 11i. It is connected to an XP512 which is being replaced by a Hitachi 9980. All the data resides on cluster aware VGs while the binaries are on local VGs. Appreciate any help in this regard.

[James R. Ferguson]

Hi Sandeep:

If this is an LVM environment and you have MirrorDisk/UX, I'd use mirroring ('lvextend') to perform the replication. When done, break ('lvreduce') the mirror.

Regards!

...JRF...

[HPUX SysAdm]

Hi JRF,

Yes this is an LVM environment and MirrorDisk/UX is installed on the server. Could you give me details (stepwise if possible) on how it can be done. Thanks for your help on this matter.

[Mancboy · Accepted solution]

Hi Sandeep,
you need to add the new discs to the server, then initialise the discs to be physical volumes, then extend the volume group to have these discs in there, then you can either pvmove the disc blocks to the disc array (fastest method) or mirror the logical volumes onto the disc array then remove the previous data from the old array

so

populate the device table with the device files:
1) ioscan -fnC disk
2) insf -C disk

create the physical volumes:
3) pvcreate /dev/rdsk/cXtYdZ (for each disc presented by the array; referred to now as newPV1, newPV2, etc)

extend the existing volume groups on the old array, with the newly created PVs
4) vgextend /dev/vgXX /dev/dsk/newPV1 /dev/dsk/newPV2 (one long line incorporating each PV you created in (3))

now you can choose which method you prefer to move the data:
5a) pvmove /dev/dsk/oldPV /dev/dsk/newPV where here you are moving the data from the old array to the new array, so do this for each PV in the old array
5b) slightly similar to above, but allows you to control which LVs you move, pvmove -n /dev/vgXX/lvolY /dev/dsk/oldPV /dev/dsk/newPV (note that as long as there is room on the destination PV, you can move as many LVs as possible)
5c) lvextend -m 1 /dev/vgXX/lvolY /dev/dsk/newPV to copy the extents to the new array and then lvreduce -m 0 /dev/vgXX/lvolY /dev/dsk/oldPV

Finally remove the LVM structures from the old array:
6) vgreduce /dev/vgXX /dev/dsk/oldPV

In the options for (5) above, I usually choose (5b) so I would find out which LVs I have on the old array first, then manually move them across to the new array. It means I am forced to check my work as I go along and also gives me a chance to spot any old LVs that any previous admins had left behind.

[Michael Steele_2]

First question to ask: Do you have single or duel HBAs connected to the XP512 on all of your servers? Since you're in a ServiceGuard environment then I'm assuming duel. The difference is substantial. Single HBA migration involves a vgexport / vgimport and will not always work, while a duel HBA procedure vgreduces all of the alternate luns out of every vg, disconnects the alternate HBA from the XP512, and then reconnects the alternate into the Hitachi, which then maps out the new luns to the server. You can then build new vgs, which is IMPORTANT to do if the NEW luns are physically larger on the Hitachi; else, your OLD vgs will be limited to the smaller size of the XP512 luns. Check the lun sizes. Refer to vgcreate, max_pe, pe_size, etc., and plan for new vgs that can take the bigger luns.

Whether or not you are directly connected to these arrays or going through a SAN switch is also important. A brocade switch, for instance, has the ability to include both XP512 and Hitachi while directly connecting to the arrays does not.

Whether going through a SAN switch or directly connecting plan on the device names of the disks to change! In the case of a single HBA and vgexporting the LVM header will still reside on the disk, you just need to clean up /etc/lvmtab and vgimport with the new device names. Once the vg is back up with the new device names, then vgcreate new volume groups with the new luns.

Either way, youâ re aiming to create temporary file system mount points to copy data into. Use the 'cp -pr' command. Once the copied, umount the temp. mount point and remount using the old mount point. Use 'cksum' and 'sdiff' to compare the temporary file system with the original file system.

Write the procedure out, its a lot of steps, and refer to a capacity planning document. In fact, you should write the capacity planning doc first planning for future growth and free vg space. When I do migrations, I target a new logical volume / file system size to be under 70%. ( USED MB / .70 = X ). Where X is your new logical volume size. I then add in another 15% to the entire vg for emergency file system extensions.

Having a monthly historical 'bdf' record of used mb is important. Some filesystems grow, so do not. Some see seasonal growth. Try to get a year's worth if possible. Failing that, get a week or a month and use sampling calculations.

[Michael Steele_2]

Couple of other things, Hitachi Disk Array Server System Requirments. For example, Online JFS ( 3.3 at least and pain to check ), HBA product numbers, and HBA firmware versions. Hitachi should provide you with these server system requirements. So plan on patching ( HWEnablement Bundle ) first.

[Michael Steele_2]

I have to disagree with Mancboy about using pvmove or mirroring. This will never work if you have larger luns because you'll be restricted to the smaller lun sizes. Also, pvmove is inherently dangerous; Its a move and not a copy. And interrupting a pvmove leads to total loss of data.

[HPUX SysAdm]

Appreciate your help and the detailed steps for doing the data migration. The server has two FC HBAs connected to the storage arrays (XP and Hitachi) through a SAN switch. I can see the device files on the Hitachi by doing an ioscan on the server.

Will the pvmove method minimize downtime and will it be able to handle files greater than 2G in size. Any help would be greatly appreciated. I am newbie to HP-UX and don't have any experience in this.

[James R. Ferguson]

Hi:

Be very careful if you use 'pvmove'. Any abnormal termination of an in-progress 'pvmove' can leave the logical volume in an unusable state.

The advantage to using a mirror replication is that you have at least one good copy of data on your old disk and one good one on your new disk. In fact, LVM mirroring allows *two* copies. Hence you can keep your old SAN logical volumes mirrored while *adding* a mirror copy to your new SAN. Then reduce one mirror copy from the old SAN and add that copy to the new SAN before finally severing use of the old one.

Regards!

...JRF...

[Michael Steele_2]

Two HBA's is ideal. vgreduce out all the luns attached to that HBA and note that there might be more than one vg. Once that HBA is free then you can attach the Hitachi. Bear in mind the larger lun problems that I've posted above. This is a big bear trap ( pun ) and you don't want to step in it by not making new volume groups. Review the Hitachi System Requirements and plan on patching first. For its entirely possible that some of your parts or their firmware versions predated the creatation of a new Hitachi disk array. Phase one, capcity planning and step by step procedure, phase two, patches, phase three, vgreduce the alternate HBA, phase four, attach the new luns from the Hitachi, phase five, new vgs ( MAX_PE and PE_SIZE in vgcreate ), phase six logical volumes, phase seven, temp file system mount points on new luns, phase eight, copy date ( cp -pr ), phase nine, verify data with cksum and sdiff, phase ten, remount new logical volumes with verified data to old mount points.

[HPUX SysAdm]

Thank you all for the wonderful replies. Greatly appreciate if you could clarify another doubt.

We are running Oracle 9i on this server. One mount point holds all the Oracle binaries while the data and index is spread out on other mount points. The oracle related mount points are given below

/dev/vgorab/orabin 9216000 5445640 3652594 60% /opt/oracle
/dev/vgora1/oradata 1048576 378692 628062 38% /d01/data
/dev/vgora2/oradata 81920000 75233648 6675640 92% /d02/data
/dev/vgora3/oraidx 2097152 1295290 752890 63% /idx01/index
/dev/vgora5/oradata 11403264 2826640 8040712 26% /d03/data
/dev/vgora4/oraidx 2097152 1095290 1001862 48% /idx02/index


lvextend is great for Oracle data, but what about the Oracle software. Do I need to re-install the Oracle software/binaries or will lvextend take care of it.

I understand that the lvextend procedure is used for making mirrored boot drives but wanted to be sure about its usage for application binaries before proceeding. Greatly appreciate the help.

[Michael Steele_2]

No problem with oracle binaries being cp'd into a new file system. The application moves right over. And if there are problems, then you still have the old luns still attached for roll back. This is a very safe procedure.

[Berd]

Could you not have both the old and new disks presented to the server at the same time. Providing you have SAN connectivity to allow this. Do your pvcreate's, volume group and lvol build. Mount on temporary mount points.

You could then stop your application at an appropriate point in time, and use dd to copy from your old disk to new. Unmount all application mount points and mount up the new logical volumes and restart your app.

This is how I migrated from XP256 -> EMC array.

HTH

Berd

[HPUX SysAdm]

Thank you all for the help. I'ave what I need to do the LUN migration.

[HPUX SysAdm]

.