topic Re: Script to create vg's and lvols in Operating System - Linux

Script to create vg's and lvols

Ray Allen_1 — Tue, 24 Jul 2007 14:57:44 GMT

Hey Guru's,

Does have a script or know of a way to script to create several vg's, and several lvol's within the vg's? I'm not a scripter, but I was looking at the "for i" loop.
I have several vg's with some vg's that have 20-30 lvols in them. Recreating them by hand in case I had to restore from backup would take entirely too long.

Thanks for your help

Re: Script to create vg's and lvols

Juan M Leon — Tue, 24 Jul 2007 15:11:40 GMT

I use this script to create multiple vg's and lvol's. Basically I have a table file with the information needed. If is not clear just let me know and I can help you.

P.S. If you like it and it works for you, please dont forget to assign some points. greatly appreciated..

Re: Script to create vg's and lvols

Juan M Leon — Tue, 24 Jul 2007 15:12:58 GMT

This is an example of the filesystem.txt file
########Schema file #######
###Volume Group Section###
###VGlabel:volumegroup:disknames###
VG:vg01:c3t0d0,c6t0d1:c6t0d0,c3t0d1
VG:vg02:c3t0d2:c6t0d2
###Filesystem Section###
###FSlable:volumegroup:mountpoint:lvolsize###
FS:vg01:/usr/sap/DEV:1572864:lvol1 :
FS:vg01:/sapmnt/DEV:1048576:lvol2 :
FS:vg01:/oracle:1024000:lvol3 :
FS:vg01:/oracle/stage:3174000:lvol4 :
FS:vg01:/oracle/DEV:6500352:lvol5 :
FS:vg01:/oracle/DEV/sapreorg:1572864:lvol6 :
FS:vg01:/oracle/DEV/sapdata6:6963200:lvol7 :
FS:vg01:/oracle/DEV/sapdata5:6963200:lvol8 :
FS:vg01:/oracle/DEV/sapdata4:6963200:lvol9 :
FS:vg01:/oracle/DEV/sapdata3:6963200:lvol10:
FS:vg01:/oracle/DEV/sapdata2:6963200:lvol11:
FS:vg01:/oracle/DEV/sapdata1:6963200:lvol12:
FS:vg01:/oracle/DEV/saparch:1048576:lvol13:
FS:vg01:/oracle/DEV/origlogB:204800:lvol14:
FS:vg01:/oracle/DEV/origlogA:204800:lvol15:
FS:vg01:/oracle/DEV/mirrlogB:204800:lvol16:
FS:vg01:/oracle/DEV/mirrlogA:204800:lvol17:
FS:vg02:/oracle/DEV/oraarch:10485760:lvol1 :

Re: Script to create vg's and lvols

James R. Ferguson — Tue, 24 Jul 2007 15:14:14 GMT

Hi Ray:

This isn't hard. I presume that your logical volumes differ in size, so you might as well just in-line your code. That is, simply write the necessary 'pvcreate', 'mkdir', 'mknod', 'vgcreate', 'lvcreate, and 'newfs' statements as needed.

For repetitive tasks you can do (for example):

for V in 01 02 03 04 05
do
mkdir /dev/vg${V}
mknod /dev/vg${V}/group c 64 0x${V}0000
done

Regards!

...JRF...

Re: Script to create vg's and lvols

Juan M Leon — Tue, 24 Jul 2007 15:17:07 GMT

My apologies I forgot to mention that on the lines that begin with VG
Fields are separated by ":" the disk devices are separated by ",". the second field is the Primary path for thet VG and the third field is the alternate path for the VG.
########Schema file #######
###Volume Group Section###
###VGlabel:volumegroup:disknames###
VG:vg01:c3t0d0,c6t0d1:c6t0d0,c3t0d1
VG:vg02:c3t0d2:c6t0d2

For the Filesystem with in the same file (filesystem.txt)
Fields are separated by ":"
Second field is the VG
Third field is the mount point
Forth field is the size in MB
Fifth field is the lvol name related to vg

########Schema file #######
###Filesystem Section###
###FSlable:volumegroup:mountpoint:lvolsize###
FS:vg01:/usr/sap/DEV:1572864:lvol1 :
FS:vg01:/sapmnt/DEV:1048576:lvol2 :
FS:vg01:/oracle:1024000:lvol3 :
FS:vg01:/oracle/stage:3174000:lvol4 :
FS:vg01:/oracle/DEV:6500352:lvol5 :
FS:vg01:/oracle/DEV/sapreorg:1572864:lvol6 :
FS:vg01:/oracle/DEV/sapdata6:6963200:lvol7 :
FS:vg01:/oracle/DEV/sapdata5:6963200:lvol8 :
FS:vg01:/oracle/DEV/sapdata4:6963200:lvol9 :
FS:vg01:/oracle/DEV/sapdata3:6963200:lvol10:
FS:vg01:/oracle/DEV/sapdata2:6963200:lvol11:
FS:vg01:/oracle/DEV/sapdata1:6963200:lvol12:
FS:vg01:/oracle/DEV/saparch:1048576:lvol13:
FS:vg01:/oracle/DEV/origlogB:204800:lvol14:
FS:vg01:/oracle/DEV/origlogA:204800:lvol15:
FS:vg01:/oracle/DEV/mirrlogB:204800:lvol16:
FS:vg01:/oracle/DEV/mirrlogA:204800:lvol17:
FS:vg02:/oracle/DEV/oraarch:10485760:lvol1 :

Re: Script to create vg's and lvols

Todd McDaniel_1 — Tue, 24 Jul 2007 16:33:15 GMT

one more thing you might take into consideration when mapping out your new Lvols.

I don't cross disk boundaries, if I can help it. Granted if I have enough disks to do it, I will.

This also gives you room for growth if you aren't allocating entire disks at Lvol creation. It can help prevent disk contention as well if you have fewer lvols on the same disks.

This is important when you may have a disk failure in the future. It will mitigate the impact of a disk failure on how many lvols are affected.

Also, when you are mapping your Lvols, try NOT to put several busy Lvols on the same disk. Spread them out over all your available disks and then add other less demanding Lvols on the same disks.

Re: Script to create vg's and lvols

Todd McDaniel_1 — Tue, 24 Jul 2007 16:38:56 GMT

Sorry I forgot one thing.

When you are creating your lvols, I normally create the LVname first then add the disks to it with another command.

I know this is a bit anal, but I like to control the disk device that is assigned to each Lvol. So that I dont cross disk boundaries.

lvcreate -n lvname /dev/vgXX
lvextend -l NNNN /dev/vgXX/lvname /dev/dsk/cXtXdX

Another trick if you are mirroring... do this.

lvextend -l 1 /dev/vgXX/lvname /dev/dsk/cXtXdX
lvextend -m 1 /dev/vgXX/lvname /dev/dsk/cNtNdN
lvextend -l NNNN /dev/vgXX/lvname

Add just one extent to the lvol then add the mirror, THEN extend it to the full size you want it to be. This will save you loads of time syncing the disks if you were to add the entire first disk then try to mirror it all at once.

Re: Script to create vg's and lvols

Ray Allen_1 — Wed, 25 Jul 2007 08:13:49 GMT

Guys,

Thans very much for all your excellent feedback. I will be testing both scripting options on my test server in the next couple of days.

James.
Is there a way to use your script to create the vg with the same minor number as the one it had before. For example: If I vgexport a vg, and I wanted to create the vg with the same minor it had.

Re: Script to create vg's and lvols

James R. Ferguson — Wed, 25 Jul 2007 08:24:04 GMT

Hi (again) Ray:

> Is there a way to use your script to create the vg with the same minor number as the one it had before. For example: If I vgexport a vg, and I wanted to create the vg with the same minor it had.

When you 'vgexport' the '/dev/vg??' directory and its contents for the volume group being exported are removed. This allows you to reuse a minor device number.

Remember that minor numbers are two-digit, hexadecimal values that (prior to 11.31) must not exceed the 'maxvgs' kernel parameter value.

Regards!

...JRF...

Re: Script to create vg's and lvols

Ray Allen_1 — Wed, 25 Jul 2007 09:50:58 GMT

Juan,

I tested the script to create a test vg and lvol. Its only creating the vg, and failing after that. Here's the post from output.

The filesystem table file

root@nhoignit:/tmp>
########Schema file #######
###Volume Group Section###
###VGlabel:volumegroup:disknames###
VG:vg03:c26t0d3
#VG:vg02:c3t0d2
###Filesystem Section###
###FSlable:volumegroup:mountpoint:lvolsize###
FS:vg03:/TEST:516096:my_test:

The script results

root@nhoignit:/tmp>./VG_Create_Script
The following Volume groups will be created :
vg03

Checking vg03 filesystem requirements
vg03 /TEST 504MB
Proceed to create the filesystmes (y/n): y
----------------------------------------------------
Creating directory and file for volume groups
vg03 does not exist, creating directory now
+ mkdir /dev/vg03
+ mknod /dev/vg03/group c 64 0x030000
+ vgcreate -l 255 -e 7167 -p 32 -s 4 /dev/vg03 /dev/dsk/c26t0d3
/dev/dsk/c26t0d3 ---->
vgcreate: Minor number of /dev/vg03/group is not unique. /dev/vg02/group has the same minor number.
+ vgextend /dev/vg03
Usage: vgextend
[-f]
[-A Autobackup]
[-x Extensibility]
[-g PhysicalVoumeGroupName]
[-z SparePV]
VolumeGroupName PhysicalVolumePath...
More arguments required.
Now we will create the Logical Volumes for vg03 ....

+ awk -F: -v volgrp=vg03 /^FS/ && $2 ~ volgrp {print $5} /tmp/My_VG_Script_Table
+ echo my_test
+ my_test
+ awk -F: -v volgrp=vg03 -v lv=my_test /^FS/ && $2 ~ volgrp && $5 ~ lv {print $3} /tmp/My_VG_Script_Table
+ head -1
lv_mount= /TEST
+ + awk -F: -v volgrp=vg03 -v lv=my_test /^FS/ && $2 ~ volgrp && $5 ~ lv {print ($4/1024)} /tmp/My_VG_Script_
Table
+ head -1
lv_size=504
+ set -x
+ lvcreate -L 504 -n my_test vg03
lvcreate: Volume group "/dev/vg03" does not exist in the "/etc/lvmtab" file.
+ mkfs -F vxfs -o ninode=unlimited,bsize=1024,version=5,inosize=256,logsize=16384,largefiles /dev/vg03/my_tes
t
vxfs mkfs: Cannot access /dev/vg03/my_test: No such file or directory
vxfs mkfs: /dev/vg03/my_test: cannot stat
+ mkdir -p /TEST
+ mount /dev/vg03/my_test /TEST
mount: /dev/vg03/my_test: No such file or directory
----------------------------------------------------
root@nhoignit:/tmp>

Also, you'll notice it creates the same minor of existing vg.

Am I doing some wrong?

Re: Script to create vg's and lvols

Ray Allen_1 — Wed, 25 Jul 2007 09:59:05 GMT

JRF,
I appologize, but maybe I wasn't clear what I asking. If I wanted to re-use the same minor the vg had before exporting it. Can that be done with the script.

Re: Script to create vg's and lvols

James R. Ferguson — Wed, 25 Jul 2007 10:09:29 GMT

Hi (again) Ray:

> If I wanted to re-use the same minor the vg had before exporting it. Can that be done with the script.

If the goal is to ascertain the minor number given only the name of a volume group, then consider this:

# VG=vgtest
# mkdir /dev/${VG}
# mknod /dev/${VG}/group c 64 0x070000
# N=`ll /dev/${VG}/group|awk '{print substr($6,3,2)}'`
# echo ${N}
07

Regards!

...JRF...
#

Re: Script to create vg's and lvols

Ray Allen_1 — Wed, 25 Jul 2007 13:21:25 GMT

JRF,
Thanks very much

Re: Script to create vg's and lvols

Juan M Leon — Thu, 26 Jul 2007 10:10:35 GMT

Hello Ray,

I was looking at the error posted, basically seems that the minor number 3 for vg03 is already in use.
The scrcipt is intended to use the same number as the VG.
for example : vg01 minor number is 1
vg02 minor number is 2
vg03 minor number is 3
I assumed that you did nto have any vgwith the exception of vg00.
If you want I can modify the script to use the next available number.

Let me know.

Thank you

Re: Script to create vg's and lvols

Juan M Leon — Thu, 26 Jul 2007 10:12:29 GMT

I was also looking at the filesystem table file is this a disk (LUN) with 2 paths or only one disk attached on a jbod?

Re: Script to create vg's and lvols

Ray Allen_1 — Mon, 30 Jul 2007 09:28:49 GMT

Juan,

Thanks for taking a look at the output, and the follow up. The server I was testing on have vg's that don't match its specific minor number. Some of them are mixed. For example:

crw-r----- 1 root sys 64 0x000000 Oct 1 2005 /dev/vg00/group
crw-rw-rw- 1 root sys 64 0x010000 Jan 20 2006 /dev/vg01/group
crw-r--r-- 1 root sys 64 0x030000 Jul 11 16:28 /dev/vg02/group
crw-r--r-- 1 root sys 64 0x020000 Jul 11 14:04 /dev/vg02test/group
crw-r--r-- 1 root sys 64 0x090000 Jul 25 12:10 /dev/vg03/group

If you can modify the script to use the next available minor, that would be great.

When you respond to this, I will assign points. I was out for a couple days last week and did not realize you had responded to my last reply.

Thanks

Re: Script to create vg's and lvols

Juan M Leon — Tue, 31 Jul 2007 13:15:54 GMT

Ray,
Apologize for the delay, here is the sccript modification. I was unable to automate the script to read the next available number. I will do it in the next script release ...=)
For now you will need to find the available minor number by listing your group file under each vg directory. and enter the available numbers in the filesystem table file.
using your previous ls -l /dev/*/group I have the following table creating vg05 and vg06. Note that at the end of the line I have used number 4 for vg05 and number 5 for vg06.
(Remember that each field is separated by ":", if you do not have information on any field you still need to place a ":" to indicate that the field is empty)

Example

########Schema file #######
###Volume Group Section###
###VGlabel:volumegroup:disknames###
VG:vg05:c4t0d0,c6t0d1:c6t0d0,c4t0d1:4
VG:vg06:c6t0d2:c4t0d2:5
###Filesystem Section###
###FSlable:volumegroup:mountpoint:lvolsize###
FS:vg05:/usr/sap/PRD:2048000:lvol1
FS:vg05:/sapmnt/PRD:1048576:lvol2
FS:vg05:/oracle:512000:lvol3
FS:vg05:/oracle/stage:3000000:lvol4
FS:vg05:/oracle/PRD:6144000:lvol5
FS:vg05:/oracle/PRD/sapreorg:2560000:lvol6
FS:vg05:/oracle/PRD/sapdata6:7041024:lvol7
FS:vg05:/oracle/PRD/sapdata5:7041024:lvol8
FS:vg05:/oracle/PRD/sapdata4:7041024:lvol9
FS:vg05:/oracle/PRD/sapdata3:7041024:lvol10
FS:vg05:/oracle/PRD/sapdata2:7041024:lvol11
FS:vg05:/oracle/PRD/sapdata1:7041024:lvol12
FS:vg05:/oracle/PRD/saparch:1048576:lvol13
FS:vg05:/oracle/PRD/origlogB:204800:lvol14
FS:vg05:/oracle/PRD/origlogA:204800:lvol15
FS:vg05:/oracle/PRD/mirrlogB:204800:lvol16
FS:vg05:/oracle/PRD/mirrlogA:204800:lvol17
FS:vg05:/opt/ops/VWR:512000:lvol18
FS:vg06:/oracle/PRD/oraach:10000000:lvol1

Please let me know how it works.

Re: Script to create vg's and lvols

Juan M Leon — Tue, 31 Jul 2007 13:22:04 GMT

Ray,

I also use this script to help me find what disk and size is available. The script reports all the SAN disks sort by HW path, then it lists disk, size and VG allocation.
If you want you can use it. the output you will get is the following

Checking 0/3/1/0/4/0
/dev/rdsk/c4t1d4: 0.28GB /dev/vg02
/dev/rdsk/c4t1d5: 0.28GB /dev/vg02
/dev/rdsk/c4t1d6: 0.28GB /dev/vg02
/dev/rdsk/c4t1d7: 0.28GB /dev/vg02
/dev/rdsk/c4t1d2: 2.08GB /dev/vg01
/dev/rdsk/c4t0d6: 9.89GB /dev/vg03
/dev/rdsk/c4t0d2: 19.53GB /dev/vg01
/dev/rdsk/c4t2d2: 19.53GB /dev/vg01
/dev/rdsk/c4t2d3: 19.53GB /dev/vg01
/dev/rdsk/c4t1d1: 19.66GB /dev/vg01
/dev/rdsk/c4t1d3: 19.66GB /dev/vg01
Checking 0/4/1/0/4/0
/dev/rdsk/c6t1d4: 0.28GB /dev/vg02
/dev/rdsk/c6t1d5: 0.28GB /dev/vg02
/dev/rdsk/c6t1d6: 0.28GB /dev/vg02
/dev/rdsk/c6t1d7: 0.28GB /dev/vg02
/dev/rdsk/c6t1d2: 2.08GB /dev/vg01
/dev/rdsk/c6t0d6: 9.89GB /dev/vg03
/dev/rdsk/c6t0d2: 19.53GB /dev/vg01
/dev/rdsk/c6t2d2: 19.53GB /dev/vg01
/dev/rdsk/c6t2d3: 19.53GB /dev/vg01
/dev/rdsk/c6t1d1: 19.66GB /dev/vg01
/dev/rdsk/c6t1d3: 19.66GB /dev/vg01

I dunno what type of SAN do you have but you will see in the script we use HITACHI. I will suggest to change thast string to your SAN hardware brand.
If you need me to adjust the script let me know.

Re: Script to create vg's and lvols

Ray Allen_1 — Wed, 01 Aug 2007 07:43:41 GMT

Juan,

Thanks very much for the updated scripts. I will test them out in the next couple of days. Super Awesome.
By the way, we have 2 EVA5K's and 1 EVA8K.

Again, thanks for all your help.