pvcreate only uses 9GB of 73GB disk

[Christian Gebhardt]

Hello

I have the following situation:

- HPUX 11.00
- Two disks in vg00 (2 x 9GB mirrored)
- Two disks in vgDB (2 x 17GB unmirrored)

The two 17 GB (unmirrored) disks should be removed with two 73GB disks (mirrored). There is no free slot in the machine and no external disk device.

How is the fastest way to do this? We can do a backup and restore vgDB but it works 7 hours (please do not laugh).

So my idea is to split the vg00 mirror, put one 9GB disk out, put the 73 GB disk in, copy the data from the 17GB's, put out the 17GB's, put the two 73 GB's and the 9GB in the machine and create both mirrors (vg00 and vgDB). Downtime is only the copy of 34 GB.

Now the problem:
- splitting the mirror vg00 --> ok
- pvcreate /dev/rdsk/c2t2d0 --> ok
- vgcreate -s 32 -e 3000 /dev/vgDB1 /dev/dsk/c2t2d0 --> ok
- but "vgdisplay vgDB1" shows:

Max PE per PV 3000
PE Size (Mbytes) 32
Total PE 271

So the new 73 GB has only the same size than the further 9GB from vg00???

Christian

[Charles McCary]

Christian,

the Max PE is what's biting you. Unfortunately, you will have to rebuild the vg in order to fix this.

c

[Dave Chamberlin]

The problem is:
Max PE per PV 3000 - this means that in this volume group, no physical volume (disk) can use more that 3000 extents. This is set at the time of the creation of the volume group. You must recreate the volume group (or create a new one) to solve your problem. When I create new groups I always specify to create a much larger number than I need (say 20000 - using the -e flag of vgcreate). That way I can replace my disks with larger ones without problems.

[Charles McCary]

Christian,

Hi - further info from man on vgcreate:

-e max_pe Set the maximum number of physical extents
that can be allocated from any of the
physical volumes in the volume group. The
default value for max_pe is 1016. However,
if the size of any physical volume exceeds
1016 times the pe_size, the default value for
max_pe is adjusted to match the physical
volume size. The maximum number of physical
extents can be a value in the range 1 to
65535.


Not sure how this pasted, but the bottom line is if you don't specify a max pe when you create the vg, it set's it to your largest disk that was used when creating the vg.



c

[Sean OB_1]

If I'm reading this correctly it all looks good. You have a max extents of 3000, and are only currently using 271.

Try creating a 18GB logical volume in that group and see what vgdisplay shows you after that.

[Patrick Wallek]

You should NOT have a problem with the Max PE per PV.

If you do the math

Max PE per PV = 3000
PE Size = 32 MB

So total available PEs = 96000 MBs

The current problem lies in the Total PEs showing, which is 271.

271 x 32 = 8672 MB.

For whatever reason you have 8672 MB showing as available.

Are you absolutely sure the disk you did the VG create on is a 73 GB disk? It appears that it is only a 9GB.

[Charles McCary]

Christian,

sorry - Patrick's right - I didn't do the math.

c

[Christian Gebhardt]

Thank you for the quick answers

Hello Sean

lvcreate -l 600 -n lv18GB /dev/vgDB1

Logical volume "/dev/vgDB1/lv18GB" has been successfully created with
character device "/dev/vgDB1/rlv18GB".
lvcreate: Not enough free physical extents available.
Logical volume "/dev/vgDB1/lv18GB" could not be extended.
Run the "lvextend" command to create space on the Logical Volume.


Size of lv18GB is 0 extents after that

Christian

[BFA6]

Hi,

If you do a diskinfo what size does it think the disk is ?

Hilary

[Christian Gebhardt]

Hello Patrick

# ioscan -fnC disk

Class I H/W Path Driver S/W State H/W Type Description
=====================================================================
disk 0 0/0/1/0.1.0 sdisk CLAIMED DEVICE HP DVD-ROM 6x/32x
disk 23 0/0/1/1.0.0 sdisk CLAIMED DEVICE SEAGATE ST318203LC
/dev/dsk/c1t0d0 /dev/rdsk/c1t0d0
disk 1 0/0/1/1.2.0 sdisk CLAIMED DEVICE SEAGATE ST39103LC
/dev/dsk/c1t2d0 /dev/rdsk/c1t2d0
disk 24 0/0/2/0.0.0 sdisk CLAIMED DEVICE SEAGATE ST318203LC
/dev/dsk/c2t0d0 /dev/rdsk/c2t0d0
disk 2 0/0/2/0.2.0 sdisk CLAIMED DEVICE HP 73.4GMAN3735MC
/dev/dsk/c2t2d0 /dev/rdsk/c2t2d0

So I think it is a 73 GB disk in the correct slot.

Hi Hillary

# diskinfo /dev/rdsk/c2t2d0
SCSI describe of /dev/rdsk/c2t2d0:
vendor: SEAGATE
product id: ST39103LC
type: direct access
size: 8891556 Kbytes
bytes per sector: 512

[Elif Gius]

Hi,
Don't trust the diskinfo output.
I had a similar confusing output.Ioscan takes the actual settings, but diskinfo often seems to remind about the old disks.

So don't let your mind get confused ;-)

[BFA6]

Hi,

Looking at the info from diskinfo, I would have to say that this is a 9GB disk

product id: ST39103LC
size: 8891556 Kbytes

Regards,

Hilary

[Pete Randall]

I'm with Hilary. That's a 9Gb disk. Are you SURE you're looking at the right one?

Pete

[Elif Gius]

Hi,
ioscan seems to show the right disk. So don't mind about "9 GB disks" ;-)

But maybe it's a problem because the disk was in the root-vg.
Think of boot disks !!!!

[Patrick Wallek]

Interesting....

ioscan shows the 73GB disk, but diskinfo shows only 9GB. And the vgcreate is only creating a 9GB volume, so there is a problem somewhere.


Hmmmmmm........

Something to try, although I don't know if it'll work.....

Use 'rmsf' to remove the device files for c2t2d0. Then do an 'insf -e' to recreate the missing device files. See if you then get better results.

[BFA6]

Hi Christian,

Has there ever been a disk at c2t2d0 ?
If diskinfo is picking up incorrect information from somewhere, maybe LVM is too ?

Hilary

[Sean OB_1]

Since you are replacing an existing 9gb drive I would remove the device files for that drive and recreate them.

man rmsf to remove the files and then insf -e to recreate them.

Then recreate the pv and volume group and let us know what you get.

[James R. Ferguson]

Hi Christian:

Did you hot-swap the disk without a reboot? If so, reboot and re-run 'diskinfo'.

Regards!

...JRF...

[Christian Gebhardt]

Hi

I did rmsf...insf...pvcreate...vgcreate before and it was the same behavour.


To James

Yes I hot-swapped the disk without a reboot, this worked on other machines and other disks well, but this is the first time I did it with a mirrored, bootable disk from vg00. Maybe this causes the problem.

I'm sorry it is not yet posssible to reboot the machine.

[Patrick Wallek]

I just re-read your question.....

Did you actually SPLIT the vg00 mirror or do you remove the vg00 mirror?

If you just did a split, try this:

1) Put the 9GB drive back in
2) re-merge the vg00 mirror
3) re-sync the vg00 mirror
4) Instead of splitting the mirror, do an 'lvreduce -m 0 /dev/vg00/lvol## /dev/dsk/c2t2d0' to actually reduce the vg00 mirrors.
5) vgreduce /dev/dsk/c2t2d0 out of vg00
6) Put the 73GB disk back in
7) Rerun pvcreate
8) Recreate your VG.

If you just split the mirrors I think there may be leftovers somewhere that know there should be a mirrored 9GB disk at that device file. Actually reducing the mirror and the disk out of VG00 should take care of any of that.

Then when you remove your 17GB drives, and put your 9GB back in, you can recreate your VG00 mirror.

[erics_1]

Christian,

I'd say James is right on this. I'd vgreduce out the disk from the vg and reboot the box. Next pvcreate -f and vgextend back in. See if this doesn't correct the issue. Let us know what happens after you get down time.

Best Regards,
Eric

[James R. Ferguson]

Hi Christian:

I trust 'diskinfo'. In fact, it is useful as a confirmation that a disk has gone sour -- you'll often get a size=0.

I can't help but wonder if the kernel isn't keeping knowledge of the old disk, hence my query.

What does 'lvlnboot -v' yield?

Regards!

...JRF...

[Elif Gius]

Hi,
sorry but I don't trust "diskinfo". I had it several times, after disk replacement that diskinfo seems to remind about an old disk...and only after the server was rebooted "diskinfo" showed the actual information.Just after a fresh reboot "diskinfo" is a good way to get information about the disk.


Christian, I didn't read that you boot the system.
So after you reduced the vg00, make a reboot. After that it shoul be no problem to create the volume group with PE more than 271.

Good luck !!!

[Christian Gebhardt]

Hi Patrick

In fact I did a vgreduce, split was the wrong word.

New Infos:

I turned out the 73GB disk from machine A and did the same procedure on machine B (same model, HPUX, Patches, ...). The difference is that the removed disk in machine B (9GB) was not in vg00, not bootable and lvols not mirrored:

- vgexport to release the old disk
- pvcreate
- vgcreate
- NO REBOOT
- vgdisplay:

Max PE per PV 3000
VGDA 2
PE Size (Mbytes) 32
Total PE 2187
Alloc PE 1089
Free PE 1098

- diskinfo

SCSI describe of /dev/rdsk/c1t0d0:
vendor: HP 73.4G
product id: ATLAS10K3_73_SCA
type: direct access
size: 71687369 Kbytes
bytes per sector: 512


It works without a reboot !!


It seems that one of the following is the reason of the problem:
- differences between vgreduce and vgexport
- disk is member of vg00
- disk is bootable
- disk has mirrored lvol's

In fact my major problem is to switch the disks, so with this new informations I think I'll do it the old way (backup, changing the disks, reboot, restore)

Thanks for your replies an I'll will you inform about new information.

Christian

[Charles McCary]

Christian, I had a similar problem with the diskinfo output and vg output and eventually had to reboot to clear. HP said that this could be related to diskinfo reading the scsi cache, but after several info. swaps with HP they could not come up with a solution other than reboot. Since I needed it up and running, that's what I did.

tx,

c