topic Re: create new vg under SG cluster in Operating System - HP-UX

create new vg under SG cluster

mrmo07 — Thu, 30 Dec 2010 06:30:10 GMT

Hi exparts,

i have rp8440 with 11iv1 in SG cluster environment with EVA8400.

currently i have one volume group name "vgvedas" with two PV running application and DB. now i have another new pv. i like to make new vg name "vgdb" with that new pv. after that i need to copy all 10 lvols (database) from "vgvedas" to "vgdb" finally i will have two vg.
how can i copy data from one vg to another vg.
what i have to do for cluster and package.

please help.

Regards,
Morshed.

Re: create new vg under SG cluster

Shibin_2 — Thu, 30 Dec 2010 08:35:02 GMT

Is that new PV is big enough / same size of vgvedas size ?

Best option, you add this new LUN to vgdevas, then mirror it to all lvols. After that split the mirror, remove your old LUNs. So, your data is in your new LUN ( PV ). Now, you may have to do vgexport and vgimport to rename this or to create a new VG called vgdb.

Another good discussion can be found here. Similar thought.

http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1293697803548+28353475&threadId=542172

Re: create new vg under SG cluster

mrmo07 — Thu, 30 Dec 2010 08:47:53 GMT

hi,

could you please explain little bit more with command line.

plzzzzzzz

Rgds/Morshed

Re: create new vg under SG cluster

Shibin_2 — Thu, 30 Dec 2010 12:04:41 GMT

You haven't answered to my query yet. Is this new PV is bigger than, the size of your existing vg vgvedas ? If it is bigger, then the following procedure will work out.

1. pvcreate /dev/rdsk/
2. vgextend /dev/vgvedas /dev/dsk/
3. lvextend -m 1 /dev/vgvedas/lvol1 /dev/dsk/
4. Repeat the above till you completely mirroring to all lvols.
5. Make sure, your all lvols have mirror copy 1 and sync status.
6. Now, break the mirror.
lvreduce -m 0 /dev/vgvedas/lvol1 /dev/dsk/

7. Repeat it till you break all mirror copies from all lvols.

8. Now, remove the disk from vg
vgreduce /dev/vgvedas /dev/dsk/

Now, you have data on your new disk.

Now, create directories and group files for your new vg.

1 mkdir /dev/vgdb
2 mknod /dev/vgdb/group c 64

Now, export your existing VG structure.

vgexport -pvs -m /tmp/vgvedas.map /dev/vgvedas

Now, import this VG structure to your new disk.

vgimport -v -m /tmp/vgvedas.map /dev/vgdb/dev/dsk/

I HAVEN'T TESTED THIS. But I believe, this will work out. Please test with at lease one small lv before doing it on all lvols.

Thanks

Re: create new vg under SG cluster

Rita C Workman — Thu, 30 Dec 2010 15:36:38 GMT

Almost...but you must remember that when you added that disk to the vgedas volume group it became a member of that vg, with the vgid existing on that disk - so that vgimport will not work.

Just simply create a new volume group like normal, with a new file system and mount it.
It is a simple matter of copying the data from one f/s to the new f/s. Or you could tar it up and untar it on the new f/s. Or you could back up the data and restore it to the new f/s.
So many options...

Then just update the new volume group into the cluster/package - clean up the old f/s -
...and your done!

Rgrds,
Rita

Re: create new vg under SG cluster

Doug O'Leary — Thu, 30 Dec 2010 16:30:14 GMT

Hey;

You can't export a single disk out of a three disk vg so the mirroring suggestion won't work as advertised.

You'll have to take an outage on the database so my suggestion would be to shut down the package then manually activate the volume groups.

You'll also want to temporarily shut down cron so that nothing kicks off automatically while you're running your copy - backups and automatic database restarts will ruin your day.

Lastly, you didn't say what flavor of database - raw/cooked filesystems, nor how large.

So, before doing anything: create a table of the lvs that need to be copied over.

/dev/vgvedas/ora_sys.dbf 500 RAW oraedas:dba
/dev/vgvedas/ora_users.dbf 10240 RAW oraedas:dba
/dev/vgvedas/oracle 2048 /oracle oraedas:dba

etc

Create the new volume group and new lvs:

# assuming lvm ver1
mkdir /dev/vgdb
mknod /dev/vgdb c 64 0x0#0000
pvcreate /dev/rdsk/${new_pv}
vgcreate -e 10000 -s 8 -i 255 -p 64 vgdb /dev/dsk/${new_pv}
vgchange -a y vgdb

cat table| while read lv size mp owner
do
echo "############################"
echo lvcreate -L ${size} -n ${lv##*/} vgdb
vcreate -L ${size} -n ${lv##*/} vgdb
done

All that can (and should) be done prior to the outage start.

When the outage starts:

1. Shutdown the database and package. (cmhaltpkg)
2. /sbin/init.d/cron stop.
3. vgchange -a e vgvedas
4. Copy the data

cat table | while read olv size mp owner
do
echo ${olv}
nlv=/dev/vgdb/${olv##*/}
echo dd if=${olv} of=${nlv} bs=1024k
dd if=${olv} of=${nlv} bs=1024k &
done

5. When the dd's finish, update raw volume ownership (if needed) and any database definitions to point the database to the new volume group. You'll want to bring the database up on the new volume group to make sure everything's working right before continuing.

If you're using oracle and raw volumes, if you're smart, your datafiles are symlinks to the actual raw volumes. That being the case, it's simply a matter of updating the symlinks. If you're using cooked or direct raw volumes, you're in for a whole bunch of

alter database rename file '${old_file}' to '${new_file}';

The key point here, though, is to make sure the database comes up on the new disk before proceeding further.

6. Deactivate the volume groups:

vgchange -a n vgvedas
vgchange -a n vgdb
vgchange -c y vgdb

7. Update the package configuration to reflect the new vgs

8. Update the package configuration to reflect the new lvs if they're to be mounted.

9. start the package

10. restart cron: /sbin/init.d/cron start

11. Go get a beer.

Even on small migrations, there's a lot of things that need to be kept in sync. Oracle, in particular, is *very* twitchy about datafiles being out of sync. If you're copying filesystems, you want to make sure no one is logging into the box and changing files while you're copying. That's also the reason to stop cron - no changes while you're copying.

wow. That's a lot of typing; but, I was bored. Everyone else is on vacation this week and there's not even service calls to do at work :)

Good luck.

Doug

Re: create new vg under SG cluster

INH — Fri, 31 Dec 2010 08:32:05 GMT

Morshed,

The simplest way of approach is here

1. Backup all configuration files on system(s) like /etc/lvmtab to /etc/lvmtab.orig
# cp â Rp /etc/cmcluster/package_file/etc/cmcluster/package_file.orig

On node 1
2. Create the physical volume
#pvcreate /dev/rdsk/cxtyd1
#pvcreate /dev/rdsk/cxtyd2
3. #mkdir /dev/vgdb
4. Create device file to new Volume group
#mknod /dev/vgdb/group c 64 0x#0000
5. Create the new volume groups
#vgcreate /dev/vgdb /dev/rdsk/cxtyd1 /dev/rdsk/cxtyd2
6. Create the new logical volumes
#lvcreate â L size_in_MB â n lvol_name /dev/vgdb

7. Create the new filesystems
#newfs â F vxfs â o largefiles /dev/vgdb/rlvol_name
8. Create the new mount point directory
# mkdir /vgdbdata ( as per your requirement)
9. Mount the new vgdb filesystem
# mount /dev/vgdb/lvol_name /vgdbdata

From here onwards we need the downtime of Database , application and cluster package.
As you mentioned in your query , assuming your esisting vg is in cluster mode
10. Halt the package(s)
#cmhaltpkg -v pkg_name

11. Activate the all cluster VGs manually.
# vgchange â a e vgvedas <- assuming this is in lcuster

12. Mount the all cluster filesystems manually
#mount /dev/vgvedas/lvol_name /current_mount
13. Copy the data from existing filesystems to new filesytesms
# cp â Rp /current_mount/* /vgdbdata
#bdf check the size of the copied is same as on old existing mount

Note: If you are running Oracle database it is advisable to use dd command. As I have personal experience that not able to copy using cp command for Oracle mounts

# /usr/bin/dd if=/current_mount of vgvedas of=/vgdbdata

#bdf check the size of the copied is same as on old existing mount
14. Once copy is completed, unmount both filesystems

# umount /vgdbdata
# umount /current_mount of vgvedas

15. Edit the cluster cntl file and made changes below.
#vi /etc/cmcluster/pakg_dir/pkg.cntl
VG[0]=vgvedas
VG[1]=vgdb

And
LV[0]="/dev/vgvedas/lv_name"; FS[0]="/mount_name"; FS_TYPE[0]="vxfs"
LV[1]="/dev/vgdb/lvol_name"; FS[1]="/vgdbdata"; FS_MOUNT_OPT[1]="-o

save and exit

16. Deactivated the new volume group and convert it in cluster mode.
# vgchange â a n vgdb
# vgchange â c y vgdb

17. Start the cluster package
#cmrunpkg â v pkg_name

On primary node
#vgexport â p -s -m /etc/lvmconf/vgdb.map vgdb

On adoptive node, copy the cluster configuration file and VG configuration

18. Copy the packge control file from primary node to adotive node
# rcp node1:/etc/cmcluster/pkg/pkg.cntl /etc/cmcluster/pkg

#vgimport -s -m /etc/lvmconf/vgdb.map vgdb

The above procedure I have implemented & on four node cluster. It worked fine.

Good luck

INH

Re: create new vg under SG cluster

INH — Fri, 31 Dec 2010 08:34:54 GMT

Posting it again as my previous reply shows some junk charectors

-------------------------------------

Morshed,

The simplest way of approach is here

Backup all configuration files on system(s) like /etc/lvmtab to /etc/lvmtab.orig
# cp â Rp /etc/cmcluster/package_file/etc/cmcluster/package_file.orig

On node 1
Create the physical volume
#pvcreate /dev/rdsk/cxtyd1
#pvcreate /dev/rdsk/cxtyd2
#mkdir /dev/vgdb
Create device file to new Volume group
#mknod /dev/vgdb/group c 64 0x#0000
Create the new volume groups
#vgcreate /dev/vgdb /dev/rdsk/cxtyd1 /dev/rdsk/cxtyd2
Create the new logical volumes
#lvcreate â L size_in_MB â n lvol_name /dev/vgdb

Create the new filesystems
#newfs â F vxfs â o largefiles /dev/vgdb/rlvol_name
Create the new mount point directory
# mkdir /vgdbdata ( as per your requirement)
Mount the new vgdb filesystem
# mount /dev/vgdb/lvol_name /vgdbdata

From here onwards we need the downtime of Database , application and cluster package.
As you mentioned in your query , assuming your esisting vg is in cluster mode
Halt the package(s)
#cmhaltpkg -v pkg_name

Activate the all cluster VGs manually.
# vgchange â a e vgvedas <- assuming this is in lcuster

Mount the all cluster filesystems manually
#mount /dev/vgvedas/lvol_name /current_mount
Copy the data from existing filesystems to new filesytesms
# cp â Rp /current_mount/* /vgdbdata
#bdf check the size of the copied is same as on old existing mount

Note: If you are running Oracle database it is advisable to use dd command. As I have personal experience that not able to copy using cp command for Oracle mounts

# /usr/bin/dd if=/current_mount of vgvedas of=/vgdbdata

#bdf check the size of the copied is same as on old existing mount
Once copy is completed, unmount both filesystems

# umount /vgdbdata
# umount /current_mount of vgvedas

Edit the cluster cntl file and made changes below.
#vi /etc/cmcluster/pakg_dir/pkg.cntl
VG[0]=vgvedas
VG[1]=vgdb

And
LV[0]="/dev/vgvedas/lv_name"; FS[0]="/mount_name"; FS_TYPE[0]="vxfs"
LV[1]="/dev/vgdb/lvol_name"; FS[1]="/vgdbdata"; FS_MOUNT_OPT[1]="-o

save and exit

Deactivated the new volume group and convert it in cluster mode.
# vgchange â a n vgdb
# vgchange â c y vgdb

Start the cluster package
#cmrunpkg â v pkg_name

On primary node
#vgexport â p -s -m /etc/lvmconf/vgdb.map vgdb

On adoptive node, copy the cluster configuration file and VG configuration

Copy the packge control file from primary node to adotive node
# rcp node1:/etc/cmcluster/pkg/pkg.cntl /etc/cmcluster/pkg

#vgimport -s -m /etc/lvmconf/vgdb.map vgdb

The above procedure I have implemented & on four node cluster. It worked fine.

Good luck

INH

Re: create new vg under SG cluster

mrmo07 — Sun, 02 Jan 2011 06:58:03 GMT

Hi INH,

still some some junk characters are here. could u please attach the file instead of pasting.

Regards,
Morshed.

Re: create new vg under SG cluster

mrmo07 — Sun, 02 Jan 2011 11:22:59 GMT

Hi exparts,

i am planning to create new vg as vgdb. then i will copy all database lvols from existing vg to new vg. only one confusion is what can be best way to copy data( safe and faster and easier).

this is 750 GB oracle 8i database on hpux11iv1.

Plz help.

Regards/Morshed.

Re: create new vg under SG cluster

Doug O'Leary — Sun, 02 Jan 2011 13:24:07 GMT

Hey;

dd the volumes. As I mentioned in my previous post, you create a table of the lvs that comprise your database, then loop through them to dd the old volumes to the new ones:

dd if=${old_vol} of=${new_vol} bs=1024k

dd will copy the volume at a bit level so any filesystem definitions will get copied as well.

So, start out with both volume groups activated, but with nothing mounted, then dd the appropriate volumes to the new vg.

Then, remount the filesystems using the new vg. Remember to update your package control file to reflect the new vg - both in the ^VG* lines and the in the section that defines the LVs and their mount points.

How you proceed from there is dependent on how your oracle datafiles are identified as there are a number of options. So, *befrore you do any migration*, identify your oracle datafiles:

sqlplus / as sysdba
col file_name format a45
col tablespace_name format a45
col member format a45

select file_name, tablespace_name
from dba_data_files
order by file_name;

select member from v$logfile
order by member;

option 1: if your oracle database is using raw volumes and symbolic links to them, update the symbolic links to point to the raw volumes in the new vg.

option 2: if your oracle database is using cooked filesystems, you should be good to go as everything is mounted in exactly the same spot.

Any other option, pretty much means you're running multiple instances of

alter database rename file '${old_file}' to '${new_file}';

HTH;

Doug

Re: create new vg under SG cluster

mrmo07 — Mon, 03 Jan 2011 10:43:48 GMT

hi,

if i like to take cold backup of database and restore into newvg instead of using dd. will it ok ?? what should be the procedure??

Regards,
Morshed.

Re: create new vg under SG cluster

INH — Mon, 03 Jan 2011 22:32:42 GMT

Morshed,
Please find the attached procedure.
copy option can be used if your Oracle data file size is not in GBs.
Else go for dd option.
more safer option is go backup and restore option once you have new vg.

Good luck

Regards,
INH

Re: create new vg under SG cluster

mrmo07 — Wed, 05 Jan 2011 13:21:57 GMT

Hi,

if i like To restore is the sequence is right?

1. take cold backup to tape ( 2 set)
2. create vgdb and same number of lvols with same size as existing lvols under vgvedas.
3.unmount all existing lvols
4.mount all newly created lvols with the same directory ( existing)
5.restore from tape to same location as Data protector will restore under mount point.

i am not changing mount point but lvols.

plz advise.

Regards/MOrshed.

Re: create new vg under SG cluster

INH — Wed, 05 Jan 2011 23:02:15 GMT

Morshed,
--------------------
i am planning to create new vg as vgdb. then i will copy all database lvols from existing vg to new vg. only one confusion is what can be best way to copy data( safe and faster and easier).

this is 750 GB oracle 8i database on hpux11iv1.
------- As said above,
1) BAckup and rstore
2) dd or copy option
As i said in my last in my last post. my personal experiance that if Oracle file size is in GB then copy will not work. So go for dd or else the most safe and easy way perfrom backup and restore.

Regards,
INH

Re: create new vg under SG cluster

mrmo07 — Sun, 09 Jan 2011 07:22:43 GMT

hi ,

thnaks all.

i have restore from tape.

it was fine.

regards,
Morshed.