LVM structures inconsistancy?

[eran maor]

Hi Mark

first i wanted to tell you that in the fstab i can see that you are an entry for /var and /usr with the same volgical volume - are this a mistake ?

second , it seem to me like you that the system doesnt know how to boot peroply .

here is a good action plan that i made for another cu that had similar to what you have :

ACTION:

1. Boot in the Maintainance Mode (hpux -lm)
2. make a "vgexport -m /dev/vg00"
3. mkdir /dev/vg00
4. mknod /dev/vg00/group c 64 0x000000
5. vgimport -m /dev/vg00 /dev/dsk/
6. shutdown -r 0 -y

and do the following:

lvlnboot -b /dev/vg00/lvol1
lvlnboot -r /dev/vg00/lvol3
lvlnboot -d /dev/vg00/lvol2
lvlnboot -s /dev/vg00/lvol2

and tell me feedback

[Mark Tiwari]

Eran,

the repeat in fstab was a typo , the bain of copy paste.

as far as:
>3. mkdir /dev/vg00
>4. mknod /dev/vg00/group c 64 0x000000

is concerned, those entried already exist in /dev... what should i do about them?

OR If I am correct, the vgexport takes them out from /dev?

I'll try this out when i get back to work later this afternoon. I just came home after toiling with it all night.


tx for the reply.


-mark

[eran maor]

Hi Mark

if you did a vgexport like i said it will delete the /dev/vg00 dir.

dont delete it with a command but you the command vgexport with a map file .

vgexport -m vg00.map /dev/vg00

check to see that he is deleting the dir .

[Mark Tiwari]

I'm back at work.. And I'm going to try out what eran suggests. But before I do that... one question. If this doesn't work.. would i still be able to boot into maintainance mode with "/" mounted? What do you think?

thanks,
-Mark

[Sridhar Bhaskarla]

Another way is to do a vgcfgrestore on the the disk c0t6d0. For this you need to have a proper backup.

Look at /etc/lvmconf directory and note the time stamps of vg00.conf and vg00.conf.old. If they are not with the recent time stamps, you can use them to restore the configuration.

#vgcfgrestore -f /etc/lvmconf/vg00.conf /dev/dsk/c0t6d0
#vgchange -a y

And see if this works,

-Sri

[Sanjay_6]

Hi MArk,

Try this,

boot into lvm maintainance mode. Do a vgchange for vg00,

vgchange -a y /dev/vg00

now do a fsck for all the LV's and see if you can mount them. If you are able to do so, just reboot the system and see if it boots.

You don't have to do any vgexport and vgimport if this works.

Hope this helps.

Regds

[Mark Tiwari]

Sri:
The old lvmconf are TOO OLD ( 4 yrs) ... Which makes me wonder if they are any good. Since I have been working here for only 2 months I have no clue when this machine was setup. Do you think I should still try that?


Sanjay:
I had tried to do vgchange before and it failed. The output is:

vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c0t6d0":
Cross-device link
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c0t5d0":
The path of the physical volume refers to a device that does not exist or is not configured into the kernel.
vgchange: Warning: couldn't query physical volume "/dev/dsk/c0t6d0":
The specified path does not correspond to physical volume attached to this volume group.
vgchange: Warning: couldn't query physical volume "/dev/dsk/c0t5d0":
The specified path does not correspond to physical volume attached to this volume group.
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "/dev/vg00":
Quorum not present or some physical volumes are missing.

using vgchange -a y -q n /dev/vg00 gives the same output.

-mark

[Sanjay_6]

Hi Mark,

I doubt you are in a position to recover anything. Your best bet would be to start fresh. Load fresh OS on the disk. I might be wrong. Wait for some more help from other forum users. I personally would have started a fresh install, as you already have a backup which is 5 minutes old before your system went down.

Hope this helps.

Regds
P.S : Keep your fingers crossed.

[Sridhar Bhaskarla · Accepted solution]

Hi Mark,

Is this system been up and running since four years and no maintenance has been done so far?. I think there shouldn't be any problem in doing a vgcfgrestore with those files.

Otherwise, you can do the thing suggested by eran

#vgexport -m /tmp/vg00.map vg00
#mkdir vg00
#mknod /dev/vg00/group c 64 0x000000
#vgimport -m /tmp/vg00.map vg00 /dev/dsk/c0t6d0
#vgchange -a y vg00

See if you can be able to mount the file systems.

-Sri

[Mark Tiwari]

Sanjay:
I've already considered the idea of starting afresh. Makes a whole lot of sense. _However_ the sys admin informs me that he has no clude where the HP-UX install media is and also that HP won't supply new install media without a new contract. So basically, If i get this running fine and good. We'll take steps that this does not happen again (mirroring, regular backups etc) but if this does not work... i'm sure the managers out here are onna have a long meeting! :)


Sri:

I taked to the sysadmin here and he says that as far as he knows the lvm config backup should be fine.

But before I do this, should I not backup the existing config ?

-mark

[James R. Ferguson]

Hi Mark:

Make sure that an 'ioscan -fnC disk' shows all your hardware as "CLAIMED".

Then, try recreating /etc/lvmtab from scratch:

# mv /etc/lvmtab /etc/lvmtab.old
# vgscan -v

If I read your post correclty, you have done a 'vgscan' but only with the existing lvmtab present. When the current file is present it is used to help construct the new one -- something that I think we may not want to do.

Regards!

...JRF...

[Sanjay_6]

Hi Mark,

You can think of creating a ignite make_recovery tape from another server and use that to restore the OS on this server. Is that a viable option. Do you have another server with the same version of OS and similar hardware considerations.

I don't think the old LVM backup is of any use in this sort of situation.

Hope this helps.

Regds

[Sridhar Bhaskarla]

You don't need to backup the existing configuration as it is not working.

Is there any reason why you are skipping me giving some points while the others are fortunate enough?.

Today it sucks here.....

See you all for the day.. No more wastage of time..

-Sri

[Sridhar Bhaskarla]

Before I go... since you have already taken the decision of a fresh install, there is no harm in trying out these little things... If you are lucky enough either of vgcfgrestore, vgscan or vgexport/import may work for you..

Bye,
-Sri

[Mark Tiwari]

Hey sri!
I can't go for a fresh install! there is no install media to install it from! :) I am going to try out your solution first and then erans.

rgds,
-mark

[Mark Tiwari]

Sri:

and about the points, the reason i skipped was because i thought your approach was the closest to a possible recovery, I was saving them for a 10 :)


have a good evening


-mark

[Mark Tiwari]

Sri:

I tried:
vgcfgrestore -f /etc/lvmconf/vg00.conf /dev/dsk/c0t6d0

no luck. output is as below:
Configuration info for Physical Volume 'dev/dsk/c0t6d0' not found in vg00.conf :
Charecter-special file for PV not supplied on command line, OR PV did not belong to Volume Group /dev/vg00 when configuration file was created.



oh well... next option Erans way i guess...


-mark

[Mark Tiwari]

James:

I had tried moving the lvmtab to lvmtab.old and then running vgscan -v as well... Didn't work either. If you care, I'll redo it to give you the command output.


tx,


-mark

[Sridhar Bhaskarla]

Am still hanging around :-)

It was a mistake in my earlier message. You need to use char device file. User /dev/rdsk/c0t6d0 instead of /dev/dsk/c0t6d0.

-Sri

[Mark Tiwari]

Sri:
This is getting wierder than i thought. I was thinking that the vgcfgrestore ought to work. But here is what i got:

vgcfgrestore -f /dev/vg00 /dev/rdsk/c0t6d0
Volume Group configuration has been restored to /dev/rdsk/c0t6d0

And then vgchange -a y gave the same output as before.

-mark

[Sridhar Bhaskarla]

Hi Mark,

What does your "strings /etc/lvmtab" say?. Does it have c0t5d0 and you don't have this disk at all?.

did you try multiple vgchange -a y and see what happened?.

As this has the configuration restored, you can actually try what JRF mentioned.

#mv /etc/lvmtab /etc/lvmtab.old
#vgscan -v

and see.

-Sri

[Mark Tiwari]

Sri:

strings is in /usr/bin and /usr is not mounted :( . Is there an alternative to using strings?

and i'll try to rescan the lvmtab again.

[Mark Tiwari]

ok here's the new vgscan output:

Creating "/etc/lvmtab".
vgscan: Couldn't access the list of physical volumes for volume group "/dev/vg00".

Following physical volumes belong to the volume group.
Unable to match these physical volumes to a volume group.
use the vgimport command to complete the process.
/dev/dsk/c0t6d0

The volume group /dev/vg00/group was not matched with any physical volumes.
Scan of physical volumes complete.

i'm clueless.
-mark

[Mark Tiwari]

Well it's time for me to go home and catch some nap before coming here to work my night shift. So I guess either we'll carry this on tommorow or perhaps in the night :). I'd like to tahnk you all guys for giving your inputs regarding this pathetic failure of system maintainance. Hopefully we'll get it up and runnign tommorow! :)

-mark