topic Re: EMC Split Mirror and VxFS in Operating System - HP-UX

EMC Split Mirror and VxFS

Scott Hanson_1 — Fri, 12 Jul 2002 11:25:05 GMT

We are mirroring and splitting EMC STD and BCV disks nightly. With our database chunks we issue a "block" of the database while the split is happening to ensure a clean cut of data. However, we consistantly get filesystem errors and corrupt files from splitting VxFS filesystems. Does anyone know how you can split an EMC device group and ensure that no filesystem occurs?

Re: EMC Split Mirror and VxFS

harry d brown jr — Fri, 12 Jul 2002 11:29:48 GMT

When you split a mirror, I believe you need to do a fsck on the fs that was split off.

When you say "BLOCK" your database, do you mean you are putting it into HOT-BACKUP mode, stopping the database, or just preventing the users from using it by stopping the applications?

live free or die
harry

Re: EMC Split Mirror and VxFS

Scott Hanson_1 — Fri, 12 Jul 2002 11:34:31 GMT

We are doing fsck on the filesystem but we consistantly get errors and sometimes get corrupted files. I want to be able to ensure that the filesystem is consistant *before* we do the split.

We use Informix so we put a block on the instance which allows users to read the database but writes are paused while the split is occuring. In addition a checkpoint is executed. This allows for a clean recovery point.

Re: EMC Split Mirror and VxFS

Jordan Bean — Fri, 12 Jul 2002 17:59:17 GMT

I hope you mean that you are blocking database IO BEFORE issuing the split, not while.

I wonder if the split is occuring before the host had a chance to flush all of its buffers.

Do a sync before splitting.

Re: EMC Split Mirror and VxFS

Rich Wright — Fri, 12 Jul 2002 20:01:43 GMT

We have the same environment. We do not have any filesystem errors.

We do the following:
block on DB
Unix "sync" command on DB server
split
unblock

Re: EMC Split Mirror and VxFS

MANOJ SRIVASTAVA — Fri, 12 Jul 2002 20:11:18 GMT

Hi Scott

Before we do a split we put the databse in backup mode on the orginal server so that there are no activities which take place for updation of data , and the split is clean , this also cut some archive logs whichare copied back after the fielsystems come up . The standard practice is to do a vgchgid , vgimport and then do a fsck on the mount points , incas eoyu need the scripts which we use I can share them too.

Manoj Srivastava

Re: EMC Split Mirror and VxFS

Joseph A Benaiah_1 — Fri, 12 Jul 2002 21:35:57 GMT

Scott,

I assume that you are using raw logical volumes for Informix and not not cooked files. You could try putting Informix into quiescent mode, issue the symmir command to split the device group and then bring Informix up into online mode.

If you were using Oracle, this is not a problem as all you need to do is put the database in to backup mode before issuing the split command.

Cheers,

Joseph.

Re: EMC Split Mirror and VxFS

Scott Hanson_1 — Mon, 15 Jul 2002 07:03:58 GMT

No, you seem to have got the wrong end of the stick. Our database splits are fine. The problem is with EMC disks which contain filesystems. Obviously those filesystems are mounted when we do the split. When we split the disks, we sometimes get fsck errors when checking the filesystems and sometimes get file corruption.
Apart from unmounting the filesystem, are there any VxFS commands/utilities I can use to prevent file loss by fsck after the split?

Re: EMC Split Mirror and VxFS

Misa — Mon, 15 Jul 2002 18:01:26 GMT

Scott,

Can you verify for us that you are using a "sync" command before you execute the BCV split? In my experience, this is key to avoiding file loss/big fsck problems.

M????a

Re: EMC Split Mirror and VxFS

Martin Johnson — Mon, 15 Jul 2002 19:53:07 GMT

It sounds like you have other processes writing to the file systems than just the database. Do an "fuser" on each file system to determine which processes have open files in the file system. You must either sync and quiesse those processes or terminate ("kill -15") them to insure all the data in flushed from the buffers to the files system. All the data must be written to the files in the file system before you break the mirror, otherwise data corruption can occur.

HTH
Marty