Boot Process interupted

[Charbel Bou-eid]

Hi,

I'm a new comer to the itrc forum. I have decided to participate after I saw some of the questions/answers and I like the way everybody interact to solve problems. Well done to all of you.

Today I have faced a problem on one HP rp5470 running HP-UX 11.00 on 2 mirrored HS internal HDDs.
The machine was up and running for about 10 days and I decided today to increase /usr, /opt, /tmp & /home sizes. So I rebooted with hpux -lm, did the lvextend on the 4 lvols and then tried to reboot. Suprisingly the machine stopped immediately after displaying "Logical Volume c 64 0x03 configured as ROOT" and before displaying "Logical Volume c 64 0x02 configured as DUMP" and sent several HPMC ("Software Failure") after that it rebooted again, stopped & rebooted at the same stage.

So I decided to boot again in Maintenance Mode, ran "vgchange -a y" (for all the VGs) = succesfull (no single error message) , "mount -a" = succesfull (no single error message), then "init 4", and the machine continued the boot process normally till the Login and users accessed the machine normally.
I did a "lvlnboot -Rv vg00" and found that the boot info is correct, moved the lvmtab and recreate it with "vgscan -v"

Do you think that one of the ioconfig files is corrupted? If so, the ioinit running in /etc/inittab is not supposed to update it? or I should move/delete the 2 ioconfig files before?
Or Do I have to invoke again "lvlnboot -d /dev/vg00/lvol2" or "lvlnboot -s /dev/vg00/lvol2"?

As of this moment, the machine is running after booting it in Maintenance Mode and invoke manually init 4. By the way I have checked inittab and looks OK.

Can someone please help?

Thanks
Charbel Bou-Eid

[Michael Tully]

I'm not sure why you would want to use 'hpux -lm' ... unless you wish to boot the system without LVM.

You should have used 'hpux -is' placed the system into single user mode. Using this method, only / and /stand are mounted. You can then use commands in /sbin to do LVM jobs.

# lvextend -L /dev/vg00/lvolX
# extendfs /dev/vg00/rlvolX

On completion you can run 'mount -a', all filesystem would then mount and then run your default run level, as defined in /etc/inittab

[Charbel Bou-eid]

Hi Michael,

Thanks for your quick reply.
You're totally right and I agree but I don't know why i have missed the -is and used instead the -lm.
But nevertheless I don't think that the problem is because of the -lm as I ahve done the same on another machine yesterday and everything went normally.

Charbel

[twang]

Since you are mirroring the filesystem, so you should have enough space on both the disks. For instance,
# pvdisplay /dev/dsk/c1t6d0
# pvdisplay /dev/dsk/c2t6d0

Check the "Free PE",
If you have online JFS, you can be able to do it online without having to reboot the box into single user mode. as follows:
# lvextend -L /dev/vg00/lvol7
# fsadm -b /usr
(size_in_blocks = new_size_in_mb*1024)

If you don't have online JFS, then you should boot into single user mode:
ISL> hpux -is

Once it puts you in single user mode,
# lvextend -L /dev/vg00/lvol7
# extendfs -F vxfs /dev/vg00/lvol7
# mount /usr
check the change,
# bdf /usr
reboot into multi-user,
# shutdown -ry 0

[Sridhar Bhaskarla]

Hi Charbel,

You are not supposed to boot to multi user level from LVM maintenance mode. Since you have booted already, I would suggest to prepare a make_tape_recovery tape before you attempt any further troubleshooting.

I believe your BDRA might be incorrect that's why the system was not booting. May be your swap volume was not contiguous or incorrectly defined. Or something might have changed during your lvextend session. Try the following to ensure the correct BDRA.

lvlnboot -b /dev/vg00/lvol1
lvlnboot -s /dev/vg00/lvol2
lvlnboot -d /dev/vg00/lvol2
lvlnboot -r /dev/vg00/lvol3

Make sure your lvlnboot -v returns correct information.

-Sri

[Charbel Bou-eid]

Hi Sri,

I've done what you have proposed although the lvlnboot -v was showing correct info (again from the maintenance mode, as it is the only mode that leads me to a shell prompt) : Same.
I even lvrmboot lvol2, lvremove lvol2 and then recreate it with -C -r options, then lvlnboot -d & -s, same problem.
I tried make_tape_recovery -Av but failed as it couldn't resolve the /dev/root (due to maintenance mode).
I deleted the 2 ioconfig files, then recreated them with ioinit -c : same.
I even tried to reload the bootlif from the OS Core/Update/Recover CD, which was successful, but couldn't boot in single-user nor in multi-user.
I even moved the vmunix and downloaded the one on the CD. I wen to delete the /stand/system, generate a new kernel, reboot : same.
I've disabled the Swap & Dump, but no luck again.

To recap, the system panics right after:
"Logical Volume 64, 0x3 Configured as ROOT",
first it gives a SYSTEM ALERT : LEVEL 12 = SOFTWARE FAILURE, then an UNEXPECTED REBOOT, continue the init (or something similar) script, display a "TRAP TYPE 15 (Data Page Fault)", then "SYSTEM PANIC : Data Page Fault", then "Error : Your System crashed before I/O and dump configuration was completed, etc..", waits for me to acknowledge the Errors, before rebooting, and ... same story all over again.

What do you advise? what should be the next step?

p.s. by the way I did a mistake yesterday, the machine is running 11i (11.11) without OnlineJFS.

Thanks for your help,
Charbel

[Sridhar Bhaskarla]

Hi Charbel,

Did you check the mirrors?. Is everything mirrored correctly?.

See if you can boot the system with no quorum mode.

ISL>hpux -lq -is /stand/vmunix

-Sri