system isn't coming up

[Ragni Singh]

Hi, I have a system which happnes to be down. While trying to reboot, its failing on primary boot with message "failed to initialize". The I try to reboot using alternate path and I am getting hte following..."/stand/vmunix - not executable" Any idea how I can bring this system back o line.

I do have the latest make recovery on site. Any help is greatly appreciated.

[Uday_S_Ankolekar]

Try to boot with old copy of vmunix, vmunix.prev while in ISL prompt otherwise make_Recovery will be the best choice.

-USA..

[baiju_3]

Hi ,

Did you do kernel rebuild before rebooting .

Break the auto boot process , and do
interact with IPL and in ISL prompt

ISL > hpux -ll /stand/system

See that vmunix and vmunix.prev ar
e listed .

Then you can try with alternate kernel.

ISL>hpux /stand/vmunix.prev

Do you have a mirror disk to boot from alt. disk ?


Ragrds,
BL.

[Uday_S_Ankolekar]

Here is the process to boot from alternate vmunix.

http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000080020889

-USA..

[DCE]

If the booting from vmunix.prev from the primary drive does not work, be sure to try vmunix.prev from the mirror (just in case of a hardware failure)

[Ragni Singh]

I think I have a bad boot disk. I treid going to ISL>, then do hpux ll /stand, I got, could't find /stand. Then I did "hpux /stand/vmunix.prev, I got :not executable".

I did see the message boot failed so I'm very much incliend to say that I have a bad boot disk. And also I may not have a good mirror or something.

How does make recovery tape work? Do I just put the tape in and reboot? Any configurations I need to do?

[Mel Burslan]

put the tape in the drive
reset the box by either a power down and power up or from GSP or CM prompt, issuing an "RS" command. (to get to GSP, at the console, hit ctrl-b)

interrupt the boot sequence when you are given the chance of 10 seconds to press any key. At this point press some key and stop.

then type search and look for the sequential media line. Note the pX number at the beginning of this line and type

boot pX

then let it go boot it. If you come to a menu which looks like hpux install menu, select install option as this will totally dump the image from your tape onto the disk. Make sure you have a good disk to install this image onto. Especially if you are suspicious that your boot disk may have died, you may want to get that one checked before doing this, or at the install time, you will have top select the mirror drive as your primary disk from now onm and dump the image on this one.

[Lee Harris_5]

Hi, If your boot disks were properly mirrored, it could just be that your AUTO file has the wrong boot command. Try interacting with IPL and issuing the command ISL> hpux -lq

This will boot with the low quorum specified which will let a system boot that has less than 51% of the disks in a volume group available.

HTH

Lee

[Ragni Singh]

okay I just restored a backup for my system from last week. I look in /stand and I see the following. What is /stand/dklm/* versus /stad/vmunix. Should I be using something else to boot from?

1024 1 /stand/dlkm
1024 1 /stand/dlkm.vmunix.prev
1024 1 /stand/dlkm.vmunix.prev/mod.d
7336 1 /stand/dlkm.vmunix.prev/mod.d/krm
75 1 /stand/dlkm.vmunix.prev/mod_register
24 1 /stand/dlkm.vmunix.prev/node.d
718160 1 /stand/dlkm.vmunix.prev/symtab
2107 1 /stand/dlkm.vmunix.prev/system
1024 1 /stand/dlkm.vmunix.prev/system.d
204 1 /stand/dlkm.vmunix.prev/system.d/kr



18749368 1 /stand/vmunix
18697520 1 /stand/vmunix.prev

[Patrick Wallek]

/stand/dlkm are dynamically loadable kernel modules. Those are loaded by the kernel when appropriate. /stand/vmunix is definitely what you want to boot from.

[Ragni Singh]

when I tried doing ISL> hpux -lq, I got cannot find /stand or /stand/vmunix.

Do I have any other choice here or go with make recovery tape. Will make recovery break teh application and stuff that was running on this system? Does it change any configuration file or anything like that? This system has NNM running monitoring some of our corporate network. Points willl be assigned as soon as I have my system online and all will be assigned points for assiting.

[Mel Burslan]

if you use tape to recover your system, it depends what it breaks and what it does not, on how your system is laid out and how the make_recovery tape was created.

if your application and data was on vg00 and you run the make_recovery command with -x inc_entire=vg00 option, then only thing you will lose is the data between now and the time the tape was created.

If your data is some other vg and those disks are safe, you should not have any problems but it all depends. Without knowing the system layout, no one can tell you if anything is going to break or not.

[Ragni Singh]

everything was on vg00, so I think its safe for me to moun the tape, power cyccle teh system and see what happens. I'll be back to let you all know soon.

[Ragni Singh]

How can I test to see if both my internal disks on this system is bad. Before running make recovery, I want to make sure I have good internal disks becasue it does complain about boot failing and unable to initialize. I can only go up to teh ISL prompt. Thanks for your time.

[Devender Khatana]

Hi,

It does seems that only one of your system disk is having problem. Both system disks never go bad at a time so it is a very rare hit.

Are the disks listed in the search output at the ISL prompt?

If yes you can use offline diagnostics ( If installed at ISL prompt)?

The reason for not booting from the alternate boot disk seems that it was not properly mirrored and somebody never tested booting from alternate disk. Was it?

What happenned after restoring from ignite?
Why you still require to restore again?

Regards,
Devender

[Ragni Singh]

I have just inherited these systems after the real admin got fired. So I am tring to figure out what was done and what wasn't.

How can I tell which disk is bad before I run ignite. When I run ignite, does it automatically pick the good disk. I want to be able to identify the bad disk.

[Mel Burslan]

ioscan -fnC disk

and look for NO_HW phrase. if you see NO_HW in the status column for any disks, those are the bad ones.

also

diskinfo /dev/rdsk/cXtXdX

command on a bad disk most probably (not 100% of the time but almost that much) will return an error message in the sense of not being able to read from the drive you requested etc. This is your second cue to identify a bad disk.

What model machine is this ? L-Class ? N-Class ? rp-XXXX ?

[Devender Khatana]

Hi,

There are not much options to look for the reliability of the disk when the system is not bootable. You can look in the search command at ISL prompt or can use offline diagnostics at ISL prompt.

While recovering from ignite you can interrupt the recovery process to select the disk you want to restore to. If you do not interrupt the process it will probably go and install on the one which was your primary boot disk earlier.

HTH,
Devender

[Bill Hassell]

Because you are not getting disk errors (they will be init errors followed with a several rows of numbers), it is likely that something has trashed the /stand filesystem. Given the circumstances, there is always a possibility that /stand was removed and if the boot disk was mirrored, the mirror will be the same. The Ignite backup tape will put things back but with a lot of restrictions such as mirroring not enabled. It also depends on the options used to create the Ignite tape. Some options will create a golden image which is a skeleton used to build a new system.

When you restore the tape, leave the mirror disk untouched. You may be able to import the disk after restoring the system and get needed files restored (ie, /stand/vmunix) and then reboot using: hpux -lq from the old mirror, thus saving a lot of time (and data).

[Mel Burslan]

Come to think of it, as Devander said and I totaly overlooked, once the server is not up, you have no way of determining what is good and what is bad. One upside though, once you power the system down totally, if necessary by disconnecting all the power cords and after waiting for 30 seconds and powering it back up, system will re-scan for the hardware during its self test and will not register any devices, including disks, which are not on-line. So if you disk is totally dead, there is no chance that your ignite image will try that one as the primary disk, but if it only had some media problems on one of the platter surfaces, there is good and bad on this one. Bad news : You do not want to operate with a faulty device and need to get it replaced as quickly as you can and system may still think that this is the primary device. Well, the good news is, the built in intelligence in the disk controller, will circumvent using of these bad blocks and will install the system for you properly.

After the system come up, you need to make media surface test someway (calling hp and letting them do that is the best way in my opinion if you have service contract).

Hope this helps

[Florian Heigl (new acc)]

can You simply boot to ISL and issue an 'ls'?

this would list files in /stand, but this "real admin got fired" scenario let's me think You won't find anything there.

Steps I'd take:

if there is a vmunix.prev listed, try a boot not to low quorum, but rather to lvm maint. mode (hpux -lm). If that is not possible,
boot the hp-ux install media and choose starting a recovery shell.

there You can look through the single disk slices and determine both if Your disks are ok and what is left of them.

If they appear to have been wiped and are consistent (like when both miss the same crucial system files), then I'd put away one of the root disks in case the company wants to have forensice done and possibly sue the ex-admin (unless he's been an ok guy and You just consider it a mistake. *gg*), and then immediately start a plain os install and in the meantime try to gather an ignite tape and data backups.

:)

[Sameer_Nirmal]

I would suggest following

1. Remove the second disk which was shown as Alternate Path from the system physically first.
2. I am suspecting the problem with your primary disk as it has lost the LVM structures most probably BDRA is corrupt. Hence it is not getting initilised.
3. Run offline diagnostics at ISL prompt.
4. Try to boot using hpux -lm ( LVM maintainance Mode ) , If it works, then you can do LVM maintainance task such as LIF install or vgcfgrestore etc....
Rememeber you to reboot using rebbot -n command from the signle user mode and boot using hpux -lq next time.

Most probably some LIF corruption is also there for alternate disk as well
But you have to recover one disk at a time.