vgchange hangs on 11iv3

[Eric SAUBIGNAC]

Torsten, I did not noticed that you said "Let's try this for a working and non-working LUN"

There is no "working LUN". The only working one is disk1, which is an internal sas IR volume.

[Torsten.]

This all does not make really sense to me ... but a write protected LUN would be a perfect reason. No idea currently what else it could be ... all connections and devices are visible, active, ... hmmm. Even if not active, the system should tell you, not hang.

The most easy way to double check the LUNs would be to use SSSU capture config script and check if all LUNs are NOWRITE_PROTECT - WRITE_PROTECT would be wrong.

[Torsten.]

Saying this, I think about another way for testing. How about starting in single user mode and trying to activate the VG manually in read-only mode (vgchange -a r vg..)?
Not really sure about, but I think the system will not try to write to the LVM header.

[Eric SAUBIGNAC]

Torsten,

same idea, same moment : vgchange -a r works well .....

Also tried to export/import the VGs, same punishment.

Well, I will capture both eva configuration

[Eric SAUBIGNAC]

booting the system in single mode is the 1st thing I did when I saw that the console was stuck just after activating vg00.

Then when I have diagnosed that lvm was the source of the problem I have deactivated autovg activation.

We are now in normal multiuser mode without vg autoactivated. And as previously said, read-only activations works well. So I will now check in more details the EVA

[Torsten.]

This proves at least that read-only access works.

Your system has a problem to write-access the LUNs.

Question is - why???

[Eric SAUBIGNAC]

Very long process to capture the configuration. All vDisk of the first EVA are NOWRITE_PROTECT

Checking second EVA ...

I am going to try to present one VG to another integrity blade which is in an other room.

[Eric SAUBIGNAC]

Did some tests with a third BL in an other room, with same OS release. It has some vg configured like other boxes.

- first, reboot, then vgchange on all EVA vg are OK.

- stop one faulty box

- present to the third host disks from a VG on the faulty box

- import the VG

- try to activate it ---> same punishment !!!!!

grrrrr

I will now try in the other side

[Torsten.]

I'm not sure about this: Did you import the working VGs to the problematic blade or the problematic LUN to the working (third) blade?

[Eric SAUBIGNAC]

sorry : poor english :-(

==> I imported the problematic LUN to the working (third) blade

I will try the other option

[Eric SAUBIGNAC]

Finally I imported the working VGs from the working (third) blade to the problematic blade and of course it works well .... :-(((((

Conclusions :

- problematic VG have only read access to their VDisks. It seems that it is not the reason since all VDisk ar not "write protected". EVA bug ?

- problemetic VG have their LVM structure corrupted. The fact that the effects are the same on the 2 blades that experimented an electrical outage --> LVM 2.x bug ?

And of course, still no answer from HP support :-(

[Eric SAUBIGNAC]

TOC dump sent to HP. Wait and see.

Tonight we have copied 1,5 To from faulty VG opened in RO to new VG. I have now to check if copy is OK.

I will tell more when HP support will give me a diagnostic

Eric

[Eric SAUBIGNAC]

And last but not least : vgcfgrestore on problematic VG works well, though vgchange still hangs.

So there is no writing problem to EVA Luns.

And, yes, the copie has perfectly worked :-) Thanks for the question ;-)

See U later

[Eric SAUBIGNAC]

Hi guys. Some news : HP support has finally reproduced the problem. We are now expecting a workaround or a fix.

This bug happens only with September 2008 11iv3 release, not with March 2008 ...

I will tell you when I will get more news

Eric

[Eric SAUBIGNAC]

Here is the workaround :

---------------------------------------------
I am still awaiting timelines from the lab but in the meantime I have a workaround for the customer.
If they export the VG and import with just two disks, disk85 and disk90. The VG then activated -a y with quorum override. The VG can then be de-activated, exported and re-imported with all 4 PVs. Now a -a y activation should succeed (but note that because the VG was activated with one side of the mirror missing a complete re-sync will be required and therefore the activation make take some time to complete, best to use -s and then use vgsync).
---------------------------------------------

By disk85 and disk90, you must understand all mirrored disks of the VG.

I will tell U when the fix will be available.

Regards

Eric

[Jean Mesquida_1]

Salut Eric !

The newly released PHKL_38983 seems to correct this behaviour : could you check for confirmation ?

Merci et a tres bientot !
Jean

[Eric SAUBIGNAC]

Hola Juanito,

Here is a mail I received on Wenesday, November 12th :

---------------------------------------------
The patch team found a problem with the patch PHKL_38983 and the lab are looking at this issue. They think they have a fix via the patch scripts and will be working to make the changes and re-submit as quickly as possible. It is sounding like there may be another couple of weeks before the patch will be out though. I can only apologize for the delay.
---------------------------------------------

So, maybe you should wait a little before applying it.

A+ ;-)

Eric

[Jean Mesquida_1]

Hola Enrique,

Bad news : HP could have mention this information and/or unpost this patch from ITRC :-(

Still waiting.

@+
Juanito

[Eric SAUBIGNAC]

Jean,

Following your post I have checked : HP says that the patch is now OK.

Thanks everybody and see you in an other thread

Eric

[Eric SAUBIGNAC]

Solved with patch PHKL_38983 :


( QX:QXCR1000843538 )
Volume group activation may hang after a system failure
if the volume group has a mirrored MWC enabled logical
volume.

[Jean Mesquida_1]

OK Merci d'avoir verifie Eric :-)
@+
JM