/dev/vg00/lvol7 device file got corrupted.

[Kannan_9]

Hi Experts,

In my HPUX 11.11 system /dev/vg00/lvol7 got corrupted which is mounted as /usr while doing some tar archive. System is a primary node on M/C Service Guard. Now I am not able to run the system commands. I have taken the Ignite backup from make_tape_recovery.

Can I recover the device file from the ignite backup? Now I am not able to access /usr, it is giving bad directory. All usr commands are not working.

While executing any command like bdf i am getting the below output.

# bdf
sh: bdf: not found

Please help me AEAP.

Thanks in advance,

Regards,
Kannan

[Patrick Wallek]

Try using commands out of the /sbin/ directory instead.

/sbin/ commands are statically linked so they don't require libraries that are normally in /usr/lib.

[Sameer_Nirmal]

As far as the recovery tape is concerned, you can recover /usr files from it.

Take the shutdown of this server and boot it from recovery tape. Other option would be boot using OS media and resore system files from it.

[Kannan_9]

Thanks Patrick, for your immediate response.

Restoring a device file itself will solve this problem or I have to restore the full /usr filesystem. What will be the best and how to do that?

If I want to restore first "mt" command itself not working becuase it is only available in /usr/bin.

Thanks,

Kannan

[Kannan_9]

Thanks Sameer for your response.

The system is a primary node in M/C Service Guard. If I have to reboot the system i have to transfer the database to other node without that i may not able to reboot the server.

Is there anyway to recover the database file itself or /usr I have to restore.

What is the steps to restore a corrupted device file alone or the /usr file system without rebooting.

Thanks,

Regs,
Kannan

[Artyom Voronchihin]

Hello!

You can recreate device files manually using mknod(1m)

Remove corrupted files lvol7 and rlvol7 and execute following commands:

#/sbin/mknod /dev/vg00/rlvol7 c 64 0x000007
#/sbin/mknod /dev/vg00/lvol7 b 64 0x000007

[Sameer_Nirmal]

what do you mean by corruption of /dev/vg00/lvol7? Is this device file present in /dev/vg00 ? Do you see any files under /usr?

If the device file is not present then create it as said by Artyom.
Check the file system by fstyp /dev/vg00/lvol7 and mount it on /usr again . If nothing there under /usr then you have to restore /usr/files from another system. Make a tar copy of /usr from another node and try to restore using /sbin/tar.

To be at safer side do following steps
You can halt the pacakage running on that node using cmhaltpkg from secondary node.
Then start the package on secondary node using cmrunpkg < pkg-name> command from secondary node.
Run cmhaltnode . Now you can run command /sbin/shutdown . You can now use recovery tape or OS media for recovery

[rveri]

Hi Kannan ,

First as Nirmal said , what do u mean by "corrupting /dev/vg00/lvol7" ,

i) Does the device file is corrupted or its not present .
ii) Does the filesystem mounted on /dev/vg00/lvol7 i.e /usr is corrupted.

# ls -l /dev/vg00/lvol7 (pls give output).

Can u execute # /sbin/fstyp /dev/vg00/lvol7

What it gives , please give the output.

And also , what changes made before it corrupted. Please give the informations further ,we can help further and guide.

Many Thanks,
RVeri.

[RAC_1]

Get the ignite tape. (latest)

Get the mt executable from other system. (Same OS version)
Put it in /usr/bin directory.
ldd /usr/bin/mt
Check what files it lists.
copy all those files into respective dirs from other system.

mt -f /dev/rmt/0mn fsf 1
tar -tvf /dev/rmt/0mn

Check the list and particularly USR dir.

mt -f /dev/rmt/0mn rew
cd /usr (or better copy it to some other dir and then move files to /usr)
mt -f /dev/rmt/0mn fsf 1
tar -xvf /dev/rmt/0mn usr

This should get you back. compare all files under /usr to that of a healthy system.

[Devender Khatana]

Hi Kannan,
It is not the device file corruption. Device file hardly gets corrupted.

First find out wheather /usr is still mounted or not. As bdf is in /usr itself which is not accessible you can use /sbin/mount command to know this. Mount without any option will listed mounted file systems and their mount times and options.

It should be mounted. Then you need to transfer the package to other node having run the cluster commands from the other node of the cluster as cluster commands are located in /usr/sbin itself.

Also list the files in /usr. A lot of commands are listed in /sbin as well which includes all LVM commands so you can easily do certain things here. I would suggest to take tar backup of /usr in other system anyhow ( Should prefer shared storage or ftp etc. if working), transfer the tar archive to the system and restore that archive here using /sbin/tar.

HTH,
Devender