Need help recovering from disk crash

[Wes O'Neal]

First, let me say that I'm not very familiar with LVM and it'll be obvious. :) Our system has one volume group (vg00) and 4 logical volumes. The volume group consists of 12-13 physical disk drives. The /home logical volume is about 60 gig in size and recently one of the disk drives in that logical volume went bad. It's only a 2 gig drive though so I'm hoping we can still recover quite a bit of the /home area. Is that possible? The system boots fine although it complains b/c it can't query the bad drive and can't mount the /home logical volume. There use to be a way to force a mount even when it's not clean but I haven't been able to find that option.

The fsck fails in the last phase saying there is disk media problems, can't read certain blocks, etc. (so it can't repair the cylinder groups). I figured it's just complaining b/c there's now a 2 gig section of the logical volume missing but had hoped I could just ignore that to recover whatever is left on the other drives.

I tried several things yesterday that may have just made things worse but I'll mention what I did anyway. I replaced the lvmtab file and tried the vgreduce and vgscan commands just trying to get the system to quit trying to query the bad drive. It no longer does that but I still can't seem to get the /home logical volume to a state where I can mount it.

Any suggestions? I've checked and it won't be that big a deal to go with our last backup but one user did have quite a few files that she had changed that aren't in the latest backup. With only 2 gig of the 60 gig total being bad, I was kind of hoping we might be able to recover her files (if we were lucky and her files are on one of the disks that are still okay).

[Sanjay_6]

Hi Wes,

Try

mount -f /dev/vg00/lvol_name /home

Hope this helps.

regds

[Praveen Bezawada]

Hi
I am not very good at this stuff, but one thing that seems possible would be

create a new volume group

vgimport the disks that were in /home onto this new volume group
vgchange and make it available
and then try to mount it.
I have done this for non root volume groups with a single disk.
This will not do any harm to the data on the disks.
Hope this helps...

...BPK...

[Helen French]

hey,

If the file system type is vxfs, then try doing a full fsck.

# fsck -o full lvname

Shiju

[Wes O'Neal]

Thanks for the quick response.

Sanjay, the "mount -f" was what I was hoping would work but that option no longer seems to work. I'm using HP-UX 10.2 and when trying the -f option it just gives me the command usage help of the mount command.

To the other response, the file systems are HFS.

I'll try the new volume group suggestion shortly.

[Roger Baptiste]

Wes,

Is this bad disk on a RAID unit?? If so, replace the bad disk and it should automatically sync up the data through RAID and get back to normal.

Otherwise, try
vgreduce -f vgname
(or vgreduce -l vgname )

That should remove the missing disk from the configuration; and just hope that you are lucky to mount the filesystem! there is no guarantee that you can have one disk containing is failed and you can restore the filesystem. Size of missing data does not matter.

So, just hope. For a longterm, have a RAID solution as a protection. With so many disks, it makes sense to have some protection against disk failures.

HTH
raj

[Wes O'Neal]

No, unfortunately, it's not part of a RAID.

I did use the vgreduce command to get the system to stop complaining about not being able to query the bad disk but that still didn't help me get the file system mounted. It basically went through quite a few bad block messages but was able to fix everything until phase 6. Phase 6 of the fsck ends with errors of not being able to read a certain block number and saying there are disk media errors, etc. File system modified but still unclean (was the basic diagnosis).

[Frank Slootweg · Accepted solution]

Hopefully this is still in time:

> Sanjay, the "mount -f" was what I was hoping
> would work but that option no longer seems
> to work. I'm using HP-UX 10.2 and when
> trying the -f option it just gives me the
> command usage help of the mount command.
>
> To the other response, the file systems are HFS.

Make sure you are using "-F hfs" when using "-f". An example of both failure (without "-F hfs") and success (with "-F hfs"):

# uname -r
B.10.20
# /sbin/fsclean -v /dev/rfloppy/c0t1d0
/sbin/fsclean: /dev/floppy/c0t1d0 not clean, run fsck
# /sbin/mount /dev/floppy/c0t1d0 /floppy
mount: /dev/floppy/c0t1d0 needs to be fsck'ed before mounting
mount: warning: if device /dev/floppy/c0t1d0 is mounted to another system
then do not run fsck
# /sbin/mount -f /dev/floppy/c0t1d0 /floppy
usage: mount [-l][-v|-p]
mount [-F FStype][-eQ] -a
mount [-F FStype][-eQrV][-o specific_options]
{ special | directory }
mount [-F FStype][-eQrV][-o specific_options]
special directory
# /sbin/mount -f -F hfs /dev/floppy/c0t1d0 /floppy
# bdf /floppy
Filesystem kbytes used avail %used Mounted on
/dev/floppy/c0t1d0 1356 9 1211 1% /floppy
#

What does *not* work anymore on 10.X and higher is the 9.X and older combination of "-f" (force) *and* -r (read-only):

# /sbin/umount /floppy
# /sbin/mount -f -r -F hfs /dev/floppy/c0t1d0 /floppy
usage: mount [-l][-v|-p]
mount [-F FStype][-eQ] -a
mount [-F FStype][-eQrV][-o specific_options]
{ special | directory }
mount [-F FStype][-eQrV][-o specific_options]
special directory
# /sbin/mount -r -F hfs /dev/floppy/c0t1d0 /floppy
mount: /dev/floppy/c0t1d0 needs to be fsck'ed before mounting
mount: warning: if device /dev/floppy/c0t1d0 is mounted to another system
then do not run fsck
#

I hope this helps.

[Wes O'Neal]

Thanks for the info. The -F hfs option was what I had been missing. I guess I didn't have to use that in the past so I was just ignore the -F options part. I came really, really close to just deleting the volume but decided to search with the lrom CD manuals and then discovered the rest of the options (once I found a generic, hfs and one other mount type command).

That ended up working. The volume was mounted and I recovered everything that I needed.

Thanks again for all the help.

[Wes O'Neal]

Sorry it took a while to assign points. I just did that today 12/20/2001.

For RajMan, even though your solution was in terms of a RAID system, I still used the solution you provided to help get the kernel and config file back in sync.