Issues with 11.11 volume group

[donna hofmeister]

Take a look at this:

VG Name                     /dev/vg01
VG Write Access             read/write     
VG Status                   available                 
Max LV                      255    
Cur LV                      2      
Open LV                     2      
Max PV                      16     
Cur PV                      3      <---
Act PV                      2      <---
Max PE per PV               2187         
VGDA                        4   
PE Size (Mbytes)            32              
Total PE                    4374    
Alloc PE                    2187    
Free PE                     2187    
Total PVG                   2        <---
Total Spare PVs             0              
Total Spare PVs in use      0              

 Clearly that Cur PV <> Act PV is "not right". The thing is I'm not sure why it's complaining. Later in the vgdisplay, I see:

   --- Physical volumes ---
   PV Name                     /dev/dsk/c4t8d0
   PV Name                     /dev/dsk/c5t8d0	Alternate Link
   PV Status                   unavailable              
   Total PE                    2187    
   Free PE                     2187    
   Autoswitch                  On        

   PV Name                     /dev/dsk/c4t10d0
   PV Name                     /dev/dsk/c5t10d0	Alternate Link
   PV Status                   available                
   Total PE                    2187    
   Free PE                     0       
   Autoswitch                  On        

Plus, vgdisplay doesn't complain that some /dev/dsk/ isn't available like I would expect to see if a drive had failed. I'll add that there are and should be two drives in this volume group. There never was a third drive.

So, short of rebuilding the volume group is there a way to fix this?

Also, this volume group has a PVG:

   --- Physical volume groups ---
   PVG Name                    vg03                       
   PV Name                     /dev/dsk/c4t8d0            
   PV Name                     /dev/dsk/c5t8d0            

   PVG Name                    vg04                       
   PV Name                     /dev/dsk/c4t10d0           
   PV Name                     /dev/dsk/c5t10d0           

 (The dreadful PVG names are not my fault

I don't see that the PVG is adding any real value and I'd like to eliminate it. Is it simply a case of removing the PVG definition file?

[Torsten.]

First I would investigate why the status is unavailable and try to make it available again.

[donna hofmeister]

well, yes...

backing up a few days, this issue (with Cur <> Act) has existed for a while -- hence my original question. Definitely, before c4t8d0 was replaced last night. afaik, c4t8d0 was lvreduce'd out of the volume group, the physical drives swapped and a pvcreate was done on the new drive. (it wasn't done by me...)

i'll add that the old c4t8d0 (which was showing signs of trouble) was a 73Gb drive. The replacement is a 146Gb drive. Its mirrored pair (c4t10d0) is also a 73Gb drive. i pointed out that the extra space on the 146Gb wouldn't be accessible...

So speaking of said problem child (c4t8d0) -- this disk as far as i can tell is "functioning". That is, it's "claimed" in ioscan and diskinfo gives good results.  So does dd. pvdisplay shows the "pv status" as unavailable but you can see that it's part of vg01.  All the extents are stale, of course.

When I try to "vgsync", I get the "Couldn't re-synchronize ..." error. (I haven't tried lvsync but I expect the results to be the same).

[Torsten.]

The reason for trouble is (as most of the times) a wrong replacement procedure.

Obviously the old disk was not removed from the VG, but now there is another new disk at the same address.

Correct procedure also depends on the hardware.

Can you give more details, please?

As a reminder:

https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c01911837

When Good Disks Go Bad: Dealing with Disk Failures Under LVM

[donna hofmeister]

(i know the "good disks" document  

11.11 and an rp3440.

[donna hofmeister]

sigh...

just for giggles ('cause we have to laugh, right?), i did another 'vgchange -a y' for this volume group *and* decided to check to see what was in syslog afterward.

...PVLink 31 0x048000 Wrong Disk!

how far do i need to backup to straighten this out? 

[Torsten.]

I assume the new disk was added to the VG without removing the old disk?

It was not just replaced (vgcfgrestore)?

Can you confirm?

[donna hofmeister]

i don't have enough "bread crumbs" from the person that attempted to replace the disk to know exactly what was done.

i do know the following are the steps this person was considering doing:

- hot swap disk
- ioscan -fnC disk (check for claimed)
- pvcreate -f /dev/rdsk/c4t8d0
- vgcfgrestore -n /dev/vg01 /dev/rdsk/c4t8d0
- lvdisplay -v /dev/vg01/lvolx  (check for stale counts to decrease)
- Only if stale extents are not decreasing: vgsync /dev/vg01

the pvcreate step isn't necessary (to the point of being wrong?) of course.

[Torsten.]

I seriously doubt about the vgcfgrestore step, because all the PEs on the new disks are empty.

Total PE                    2187    
Free PE                     2187   

You could check that with pvdisplay -v /dev/dsk/c4t8d0

[donna hofmeister]