Best method of migrating data from an EMC to an EVA5K SAN

[Khalil Ahmed]

Question: I would like some suggestions as to the best approach to carrying out a data migration on our production system, which will involve the least amount of downtime.

Here is what we are trying to do. We have N-4000 Sever that is our production database server. The N-4000 is currently attached to an EMC disk array. The root VG is obviously on internal disks, all other VGs are on the external EMC disks, and there are about 30 VGs configured.

Next week a new EVA5000 disk array will be arriving. This EVA5K SAN will ultimately replace the old EMC disk array. However initially the N4000 will be wired up so that it can see both the EMC and the new SAN.

Here is what I was planning to do, see if you agree. I was going to create new VGs, plus associated LVs on the N4000 to duplicate the existing VG configuration, while the N4000 is still in a production environment. I was going to make sure that the sizes of the LVs in the new VGs are the same as the LVs in the existing VGs. Of course I will have to give the new VGs different names, for example the existing VGs are named vg08, vg09, vg10 etc and the new ones will be called vg008, vg009, vg010 etc.

During the next step I was going to create the mount point for the LVs using again different names from the live setup. Then using a recent off-line backup of our production server I was going to restore the data to the new LV mount points.

Down time would then be required only for the last step, which would be while switching to the new VGs and then doing a â Roll Forwardâ of the database by applying the database redo logs.

Thatâ s it! If any of you out there have had experience of doing this and know of a better way, please kindly share your thoughts.

By the way, if you think my method is okay can anyone tell me how I can quickly find out (through a script possibly) the MINOR numbers that are already being used by the existing VGs, (without of course having to â cdâ into each and every VG directory).

Many Thanks

Khalil

PS: I hope I have posted this question to the right forum, if it isnâ t and you think it should be on another forum please let me kno

[harry d brown jr]

If you have mirror-ux, AND your eva disks are the same size as your EMC disks, then you can MIRROR the LV's, break the EMC LV's off, then you are on the EVA.

Otherwise, a copy of some sort will be involved.

live free or die
harry

[Alzhy]

If you have MirrorDisk/UX AND your existing VG's have large EMC LUNs AND your total LUNs (aka disks) comprising your existing VG's are not in the hundreds - then you will be better off just mirroring them OR use pvmove (to larger LUNs).

Here's how I would do it so downtime is very minimal and so you can transition to EVA5K at your leisure:

1. Patch your system for SecurePath 3.0C/D
2. Install SecurePath (if 3.0D, reboot is not required)
3. If EVA already has presentation LUNs, you should see the EVA and the LUN(s) by doing a "spmgr display"
4. Allocate EVA LUNs -- either a 1 for 1 alocation (for mirroring) or larger LUNs (pvmove)
5. Once the LUNs are brought into the VGs (pvcreate, vgextend).. you have the option of doing mirrors (lvextend) or PV evacuation (pvmove)... all non-distructive..
6. Once all the LVOLS are synched up or moved... remove the EMC LUNs from each VG..

[Paul Hawkins]

Hi,
I have just migrated dozens of servers from old_array -> new_array. The quickest way by far is to recreate the vgs and then copy data using the command: cd /orig_mnt_point ;find . â depth â print |cpio â pdum /new_mnt_point
I tested this using cp, mirroring and store/restore...cpio was the fastest. There were problems using cp with large files...

[Khalil Ahmed]

Harry/Nelson, sounds really simple, and I presume what you are suggesting will work with the EMC and EVA5K configured as described below:

The EMC is configured using mirroring (raid 1 I think) and so takes care of it's own disk mirroring. As for the EVA5K SAN, that will arrive from HP with its disk groups configured using raid 5 (striping with parity checking I believ). As for mirror-Ux yest we do have it but it is used to mirror the internal root disks.

Paul, not so convinced about your approach, as this is a database server and I would have to bring the database down before doing th cpio to ensure that I have a consistent copy of the datase.

Khalil

[Geoff Wild]

I've done EMC migrations - fastest method to copy is:

vxdump -0 -f - -s 1000000 -b 16 /zmnt/oracle | (cd /oracle ; vxrestore rf -)&

No downtime way: mirroring. If your existing vg's have room, just ad the pv's from your EVA, then add them as mirrors, then remove the EMC from the vg's - 0 down time!

Rgds...Geoff

[Alzhy]

Khalil,

The LVM/MirrorDisk mirroring that we are talking about is not for protection but more so you can have copies of your data on 2 or more disk locations. It can also be a facility for split mirror backups.

How large are your Oracle Filesystems? If it is less than 1 TB and you can afford at least 8 hours of downtime, and your EMC is also FC Attached, then you can do a cpio/vxdump copy as your other alternative.

[harry d brown jr]

Khalil,

If you are using mirror-ux for your internal drives, then your server is covered to use it on other storage devices connected to your server.

live free or die
harry

[Sridhar Bhaskarla]

Hi Khalil,

I also support the mirror/UX method provided if it allows you. Most of the times you will be limited by max PV parameter. So, check to see if you can double the number of PVs in each volume group.

For me using offline-backup would be the disaster recovery plan. If you can't do mirror/ux which involves no downtime (unless the system is in serviceguard), you can replace your last step with cpio/vxdump/cp.

-Sri

[Rita C Workman]

Did a data migration from an old array to a new array last year...found this method worked fairly well for me.

Even though your going to different arrays, the principles are still there.

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=51818

By doing the transfer hostbased...I could confirm and control everything to my liking. Everything went clean and neat...and everything came back up. Was able to make the vg changes wanted. I frankly didn't want to go the vendors' method - but that's just my opinion.

HTH,
Best Wishes on your Migration !
Rita

[Rita C Workman]

Forgot to mention...on that minor number thing.

Just do a ll /dev/*/group | lp -d

Of course it does come in handy to create a spreadsheet for all your systems/pkgs so you can keep track of minor numbers.

Just a quick thought,
Rita

[Khalil Ahmed]

Thanks everyone for your recommendations, I now have enough information to decide what approach I will take ie either the direct disk copy or the mirrorUX method.

I will be assigning the points tomorrow, so once again thanks.

Regards

Khalil

[Hein van den Heuvel · Accepted solution]

You may want to review the need for many VGs.
If they were there to be able to 'spread the load' then maybe just let the EVA do that?

Instead of present dozens of smaller, disk like, LUNs maybe just present a few larger luns? Like in the 150 GB - 600 Gb range perhaps? The EVA will know how to spread it al the while maintaing redundancy sub-group.

Then carve of the large VG into the 'normal' LVs you had, or just turn the old mountpoints into subdirectories for a larger mountpoint?

Minimizing interruption is a good thing; easy of transition is a good thing; but all along you have to keep your eye open to see if you are making good use of the new stuff!
You don't want to old (trusted) ways to lock you out of new technology.

fwiw,
Hein.

[Khalil Ahmed]

Hello Hein, I read your reply this morningâ ¦ very interesting, what you say makes sense. At the moment we have many VGs because we run a SAP system with many Oracle database tables & indexes and the tables and indexes must reside on separate disks. However, with the EVA configured as Raid 5, a Disk Group will simply present us with a chunk of disk space that is in reality physically spread across many disks. Hence there is no point in having so many VGs with separate LVs per database filesystem.

Our database is currently at around 400Gbytes, so based on what youâ re suggesting we could create 1 VG, assign only 1 or 2 large LUNs to it, then create only one LV on this VG, and create all the old mount points as subdirectories under this single LV. I could then use cpio or vxdump to do a disk copy from the EMC to the new EVA. This of course would involve some down time, but still worth doing.

Can anyone think of any downside to me taking Heinâ s approach i.e. using one VG with few LUNs instead of many VGs with many smaller disk like LUNs?

Regards

[Geoff Wild]

LUNS, here's what I did.

Last EMC migration, from old frame to DMX1000.

Used to have hundreds of 8GB LUNS...each in a lvol but only a few vg's. Now, we use 32GB Metas, the frame is striped amongst 120 x 146GB physical drives. We have some 8GB meta (for redo logs).

I have 3 volume groups, 1 for database (1.3 TB), 1 for misc data/binaries (238GMB), and 1 for redo/mirror logs (2GB).

I dropped from 302 sapdata lvols to 10.

Performance is great, mounting is a lot quicker.

The separate vg's gives us flexability with DR - as we use SRDF from one city to another.

So yes, you can consolidate the number of vg's - but I wouldn't consolidate down to a single...

Rgds...Geoff

[Khalil Ahmed]

Thanks Geoff, I may adopt a similar configuration.

Just one thing though, would you mind explaining some of the reasoning behind your configuration. For example, why did you use the 32GB LUN sizes (and not much larger ones), why 10 lvols for the sapdatas etc. If I have some idea of your reasoning it may help me when I do mine.

Thanks

Khalil

[Hein van den Heuvel]

Going al the way to 1 huge volume may be too much. Some middle ground. For example, I still like to do a little bit of the good old index-data seperation, but no longer at the disk level but at 150 - 500 GB group levels. Reasons against a single big lun

- Some tools (dump/restore) like to work at a mountpoint level and a little granularity is welcome fro those.

- multiple paths: hba -> fibre-chan -> controller port path. (Either with secure-path or selective presentation). For now (untill hpux learns to pick from multiple paths like Tru64 did), hpux will just use one path per lun. So I'd want at least a lun per hba, and probably one per controller port per hba. That quickly drives you to 4 - 8 luns / eva.

- Just a little more visibility with standard systems tools like glance as to what storage area is hot and getting hotter.

- Seperate groups for redo / archiving. Like Geoff suggested, I create some minimal size (8 Disks!) groups for those to give an optimal environment the the linear sequential writes those files need.
I 'donate' the excess space (if any) to 'slow' storage like kits directories, old logfiles, whatever.

The old SAP recommanendation for sapdata1, sapdata2... is based on old storage configs with 1 mountpoint per disk. We have better tools now. I run SAP benchmarks for a living and stopped following that advice 5+ years ago.


hth,
Hein.

[Khalil Ahmed]

Thanks Hein for the further food for thought. Fortunately, I have a weeks grace before the EVA arrives, so will use the time to do my planning.

Regards

Khalil

[Sridhar Bhaskarla]

Khalil,

Having few bigger LUNs can have significant impact on the performance irrespective of what you have on the back end. All the explanations about cache on the storage, it's ability to manage the load, raids on it don't matter. System can only queue up certain number of requests (queue depth) per disk (LUN) it sees. So, it cannot automatically pump more requests if the LUN is bigger. You will need to alter 'queue depth' using kmtune (with 11i) or scsictl (11.0). Particularly if you have 11.0, there are quite a few limitations to altering this parameter.

Been there done that with bigger LUNs. I do have a 3 TB database. Now, the maximum LUN I have is only of 36GB with four paths to each LUN. It may be bit painful to manage. But I am paid for it and that occasional pain doesn't matter to me much if it is fruitful to the business.

-Sri

[Geoff Wild]

32GB LUNs was a performance/managabilty issue - we found with larger LUNS, performance wasn't as good as 32GB...we were also moving from 8GB hypers - so there was some concern (purely vapour) going to larger LUNS...10 Lvols for SAPDATA was purely for administration reasoning - also for Glance it is nice to see which lvols are busiest and it made the DBA's happy.

The 10 lvols is nice - cause each is now made up of 4 x 32GB metas.

Also, the frame is not dedicate to 1 server - and some of our other ones only require 100GB of storage (with shared apps -some as low as 30GB).

The other thing we did - was went strictly mirroring on the frame(s) - no RAID 5!

Rgds...Geoff