Are Live Split Mirrors Possible (Common) on LVM+OJFS?

[Alzhy]

On Solaris environments where we employ Vertias Volume Manager and Filesystem, we employ split mirror backups for backing up our Oracle environments. Such is possible by placing the oracle instance in hotbackup mode, then placing it in "suspend mode", invoking the mirror split commands and resuming the instance, turning off hotbackup mode -- a half TB instance done in less than 30 seconds. Netbackup also has scripts built in as it interacts with VxVM/VxFS.

Would anyone share how this would be (correctly) done in HPUX? Our experience thus far with LVM + MirrorDisk/OJFS is with mirroring the boot disks.

[Ian Dennison_1 · Accepted solution]

Nelson,

We evaluated this on our SAN-based system, but found that the usage of LVM loaded down the Fibre Channel SAN unnecessarily.

We did a few trials and found that it did work, but the preference was for using the internal disk tools on the Disk Array (Business Copy in the case of XP512, and BCVs on EMC).

Note: We insisted on 1 offline backup per week, to aid recovery if the Offlines had problems. We also found that we had to implement extensive checks on Online Backups to ensure that it did work.

What sort of disk array are you using?

Share and Enjoy! Ian

[Ian Dennison_1]

Further info,

lvsplit will reduce the mirror set of the lvol and create another lvol on the same vg with a suffix appended (usually "b"). You can then fsck it and mount it on an alternate path.

man lvsplit, it has some good worked examples on how to do this.

Share and Enjoy! Ian

[Alzhy]

Okay.. I think there should be "recipes" already out there on how to properly do splits on live oracle mount points.

On our SUN environments, we use a Hitachi 9960 (equiv. HP XP512 ?). We employ no tools on the Hitachi and entirely use Veritas products for the merging (re-synching) and the splits which does not really strain our FC infrastructre and very little impact on the DB's performance. We use split mirrors not only for backups but for moving a copy of the production instance mount points to different servers for testing and batch processing. With a SAN and even non-SAN XP512 (with multipathing) the split mirrors can be "moved" to its own diskgroup (VxVM equiv of a vg), deported from the prod server and imported on another server where backups can be done or additional tests/processing takes place...

[Ian Dennison_1]

Nelson,

I am struggling to understand if we have provided a solution to you yet.

Are you after the process steps involved in performing online splits for Oracle and HP-UX, the commands involved, etc, or are you looking for feedback on other SAs best practices for managing Split Mirror Backups?

Can you elaborate? Thanks, Ian

[Enrico P.]

Hi,
The lvsplit command splits a single- or double-mirrored logical volume into two logical volumes. A new logical volume is created containing one copy of the data.
Es:
# lvsplit -s backup /dev/vg00/lvol10
This splits the mirrored logical volume /dev/vg00/lvol10 into two copies with the new volume name /dev/vg00/lvol10backup.
The new logical volume must be checked with the fsck command before it is mounted

# fsck /dev/vg00/rlvol10backup
# mount -o ro /dev/vg00/lvol7backup /backup

After backup:

# umount /backup
# lvmerge /dev/vg00/lvol10backup /dev/vg00/lvol10

Enrico.

[Alzhy]

Am actually looking for info, best practices, tips on split mirror backups as it applies to Oracle instances and if this is possible (and common) practice.

[Stuart Abramson_2]

Nelson:

All of the high performance disk vendors have built-in "Timefinder/Business Continuance Volume" capabilities:

EMC - Timefinder/BCV
HP XP/512 - Business Copy
Hitachi Data Systems - Shadow Copy

They are very high speed, and in EMC's case use less disk.

The technique that we use (HP servers with EMC disks) is the SAME as you use:

Put Oracle DB tablespaces in hotbackup mode
Split BCV mirror.
Put oracle back in "normal mode" (reapply logs) on production server.
Present BCV mirrors to backup server.
Take backup from backup server.

But we do it internally to the EMC.

We use Legato NetWorker, but Veritas NetBackup has hooks to Timefinder/BCV ("the Array option").

Talk to your Veritas people. They probably have hooks into HP-UX mirror disk.

But, Shadow Copy, would be more efficient, and more expensive.

[Ian Dennison_1]

Aha, in that case I think I have experienced every possible failure known to Oracle!

Server Layout - Everytime you perform a remsh to perform an action on a remote server, there has to be complex checks to ensure the action was performed.

If you are checking logfiles to ensure actions are done, you need to have a check for time (are we using an old copy of a logfile which did work, rather than the latest which did not?). Our solution was to put the epoch time in several keyword messages throughout the split process. The checks performed verified that the keywords were all present for all stages of the split, PLUS, check that the epoch value was within 900 seconds of current time (15 mins).

We got caught by rcp'ing the log file back when the remsh failed, and all the keywords were present and correct, but for yesterdays commands.

Same checks for dropping out of Online mode for Oracle Tablespaces after the split; time and validity need to be checked.

Hope this is of some help. Ian