On disk LVM metadata structures are corrupted

[vandevoort]

Hi strange  thing happend twice at a customer side of  me :

the  dit via DRD a update 11.31  patch kit 0909 to 1109 .

after reboot and start up a database disk who is  not involved in this  at all , can not mount anymore :

" On disk LVM metadata structures are corrupted"

vgexport -p -m /tmp/vg.map vgsapxxx   en vgimport -m /tmp/vg.map vgsapxxx /dev/disk/diskxxx

all succesful

vgchange  :

vgchange -a y vg05PS1d1                                  
vgchange: Couldn't activate volume group "vg05PS1d1":
On disk LVM metadata structures are corrupted.

some minutes later a collegue logs  in in same machine  does a vgchange -a y vg    successful .

He did not execute anything else  .

This happeens  already on two different systems  during  upgrade our steps  last week , seems  that the operator  can not activate , and the  collegue  can some minute s later ???

looked  at the  history , no additional commands are  executed in the  time  between ???

anyone  any idea what this  could  be ..

[vandevoort]

We update a number of systems Vpars and single Blade HP-UX systems .

From patch kit sept 2009 to sept 2011.

It seems in the past  in 2009  upgraded  a lot of volume groups from Version 1.0 to 2.1  ( dynamic extend of a lun possible with this newer Version).

This seems to works fine for the past 2 ½ Year. periodical a vgcfgbackup was made of all Volume groups .

But now during Installing the patch bundle (sept 2011)after first reboot  we encountered this problem ..

vgchange -a y vg05PS1d1                                  
vgchange: Couldn't activate volume group "vg05PS1d1":
On disk LVM metadata structures are corrupted.

I found in the Release Guide ,

 After the migration of a volume group from version 1.0 to 2.x using vgversion, the volume

group activation fails with the following error message:

$vgchange –a y vgname

Coundn't activate volume group vgname

On disk, LVM metadata structure are corrupt.

Resolution notes: vgversion has been enhanced so that volume groups migrated using the

March 2012 version of vgversion will successfully activate. Volume groups migrated using

older unfixed versions of vgversion may continue to face this problem. See notes against

QXCR1001182973 under “Known issues” (page 6) for more details.

Defect ID: QXCR1001182973

2.x VG fails to activate with “On disk LVM metatdata structures are corrupted” error message

Problem: Volume groups that were migrated from 1.0 to 2.X using a version

of vgversion prior to the August 2011 web release or using the

September 2011 release may fail to activate, resulting with the error

message: “On disk LVM metadata structures are

corrupted”.

Corrective Action: To fix this problem, restore the volume group back to its 1.0 version

configuration using the vgcfgrestore command, and then migrate

to 2.x using the new enhanced vgversion command, first made

available in the August 2011 web release (after September 2011

release) and also provided in the March 2012 release.

This could be the issue .

the  odd  thing in this is that we do have this on only a few volume groups  ( from resent about  10 systems updates , with each about 10 or  more  volume groups  only had  3  Volume groups  with this  problem ).

while  on regular base  a vgcfgbackup is  made , we do  not have a config for  these volume groups   while they were V1.0 anymore , overwritten by vfgcfgbackup.

Our question / request is  :

1  is  there a way  we can easily Check  ( before we start the  update ) which volume group  could have this  issue  ?

2  because vgcfgrestore is not an option anymore the  V1.0 config is  already overwritten by the  periodical vgcfgbackup of last two years  , is there another way  except copy each volume group to a new created 2.1 volume group before update the system patches ?

the problem in this  with the solution in  2 given is  that this could  be more than 100 Volume groups  and will take months of work  a lot of temporary extra storage  and unacceptable  amounts  of disruption of the production systems .

the  steps executed  above (2)   for just one 150 GB volume group took us more than 4 hours.

Does have anyone a alternate advise / or better solution ..?