SD32000 with IO chassis power failed

[stephen peng]

dear all,
i've got such problem yesterday. a SD32000's partition 1 HPMC because of one of its IO chassis power failed. the partition was consisted with two CELLs and two IO chassises. before i got the new IO chassis to replace the failed one, i try to reset the partition and boot the partition with the root disk in the disk enclosure which connected to another IO chassis, but it failed and told: bad LIF magic.i wonder whether it was the alternate root disk problem that made the partition unbootable, or it the partition was not bootable if one of its IO chassis failed, even though there was still one IO chassis funtioning.
the lifls output of the two root disk are as following:
#lifls -l /dev/dsk/c0t3d0
volume ISL10 data size 7984 directory size 8
filename type start size implement created
===============================================================
HPUX -12928 584 848 0 02/05/10 16:25:44
ISL -12800 1432 306 0 00/11/08 20:49:59
AUTO -12289 1744 1 0 00/11/08 20:49:59
PAD -12290 1752 1580 0 00/11/08 20:50:00
LABEL BIN 3336 8 0 03/04/16 02:13:06
#lifls -l /dev/dsk/c8t3d0
volume ISL10 data size 7984 directory size 8
filename type start size implement created
===============================================================
ISL -12800 584 306 0 00/11/08 20:49:59
AUTO -12289 896 1 0 00/11/08 20:49:59
HPUX -12928 904 848 0 00/11/08 20:50:00
PAD -12290 1752 1580 0 00/11/08 20:50:00
LABEL BIN 3336 8 0 01/05/30 17:53:22
ODE -12960 3344 848 0 01/05/30 17:53:25
MAPFILE -12277 4192 128 0 01/05/30 17:53:25
SYSLIB -12280 4320 352 0 01/05/30 17:53:25
CONFIGDATA -12278 4672 228 0 01/05/30 17:53:25
SLMOD2 -12276 4900 140 0 01/05/30 17:53:26
SLDEV2 -12276 5040 136 0 01/05/30 17:53:26
SLDRV2 -12276 5176 175 0 01/05/30 17:53:26
SLSCSI2 -12276 5351 116 0 01/05/30 17:53:26
MAPPER2 -12279 5467 142 0 01/05/30 17:53:26
IOTEST2 -12279 5609 363 0 01/05/30 17:53:26
PERFVER2 -12279 5972 125 0 01/05/30 17:53:27
PVCU -12801 6097 64 0 01/05/30 17:53:27
SSINFO -12286 6161 2 0 01/05/30 17:53:27
and /dev/dsk/c8t3d0 is the alternate root disk.

thanks for your reply.

stephen

[Mridul Shrivastava · Accepted solution]

1. Boot from the path of the CORE (INSTALL) media and do not interact with ISL.

Type bo ; then respond with "no" to interact with ISL

2. Select [ Run a Recovery Shell ]
Use the and/or arrow keys to navigate through the following menus,
and use the key to select an item. If the menu items are not
clear, select the "Help" item for more information.

3. Would you like to start up networking at this time? NO

4. The HP-UX CORE MEDIA RECOVERY MENU will be displayed:
s. Search for a file
b. Reboot
l. Load a file
r. Recover an unbootable HP-UX system
x. Exit to shell
c. Instructions on chrooting to a lvm /(root).


Select one of the above: r <-----

5. The HP-UX Recovery MENU will be displayed:
a. Rebuild the bootlif (ISL, HPUX, and the AUTO file) and install
all files required to boot and recover HP-UX on a target
root file system.
b. Do not rebuild the bootlif but install files required to boot
and recover HP-UX on the root file system.
c. Rebuild only the bootlif.
d. Replace only the kernel on the root file system.

m. Return to 'CORE Media Main Menu'.
x. Exit to the shell.

Use this menu to select the level of recovery desired.

Selection: c <-----

6. The BOOTLIF PATH VERIFICATION MENU will be displayed:

This menu must be used to determine the path to the bootlif (ISL, HPUX and the AUTO file).
When the information is correct, select 'a'.

INFORMATION to verify:
Path to the bootlif is 10/0.6.0

Select one of the following:
a. The above information is correct.
b. WRONG!! The path to bootlif is incorrect.
m. Return to the 'HP-UX Recovery MENU.'
x. Exit to the shell.


Selection: a <-- If CORRECT
If the address is INCORRECT then select "b" and
follow the instructions to enter the correct
hardware path of the boot disk.

7. The BOOT STRING VERIFICATION MENU will be displayed:

This menu must be used to verify the system's boot string.
When the information is correct, select 'a'.

INFORMATION to verify:
The system's boot string should be:'hpux -lm (10/0.6.0)/stand/vmunix'

Select one of the following:
a. The above information is correct.
b. WRONG!! Prompt the user for the system's boot string.

m. Return to the 'HP-UX Recovery MENU.'
x. Exit to the shell.

NOTE: For an LVM '/'(ROOT) the '-lm' option MUST be specified
(example: 'hpux -lm (2.3.4)/stand/vmunix' )

Selection: a <-----

8. You will see the bootlif installing.

9. The RECOVERY COMPLETION MENU will be displayed:
Use this menu after the recovery process has installed all requested
files on your system.

Select one of the following:
a. REBOOT the customer's system and continue with recovery.
b. Return to the Support Media's Main Menu.

Selection: a <-----

10. The system will autoboot into maintenance mode. Here you should verify
the BDRA.

11. vgchange -a y /dev/vg00 (activate the root volume group)

12. lvlnboot -v /dev/vg00 --> This should display something like what is shown
below. This sample has a mirror boot disk.

Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
/dev/dsk/c0t6d0 (10/0.6.0) -- Boot Disk <--- Primary boot disk
/dev/dsk/c0t5d0 (10/0.5.0) -- Boot Disk <--- Mirror boot disk
Boot: lvol1 on: /dev/dsk/c0t6d0
Boot: lvol1 on: /dev/dsk/c0t5d0 <---------- Mirror boot disk
Root: lvol3 on: /dev/dsk/c0t6d0
Root: lvol3 on: /dev/dsk/c0t5d0 <---------- Mirror boot disk
Swap: lvol2 on: /dev/dsk/c0t6d0
Swap: lvol2 on: /dev/dsk/c0t5d0 <---------- Mirror boot disk
Dump: lvol2 on: /dev/dsk/c0t6d0, 0

After booting the system the BOOT, ROOT, SWAP,and DUMP volume information
should be correct. If something is missing use the below commands to add.
For example:
BOOT: "lvlnboot -b /dev/vg00/lvol1"
When '/stand' is ONLY on a separate logical volume.
If your not sure check /etc/fstab for a seperate lvol
for /stand.
Otherwise BOOT and ROOT are considered identical.

ROOT: "lvlnboot -r /dev/vg00/lvol3"
Or lvol1 if their is NOT a seperate lvol for /stand

SWAP: "lvlnboot -s /dev/vg00/lvol2"
DUMP: "lvlnboot -d /dev/vg00/lvol2"

13: If you had to make any changes then verify the information is correct with:
"lvlnboot -v /dev/vg00"

14: enter "lvlnboot -R /dev/vg00"

15: Then the boot string should be changed back:
"mkboot -a "hpux (;0)/stand/vmunix" /dev/rdsk/c0t6d0" <-- Primary disk
"mkboot -a "hpux -lq (;0)/stand/vmunix" /dev/rdsk/c0t5d0" <-- Mirror disk

16. Reboot the system
with "reboot"

[stephen peng]

hi Mridul,
i do know it is a way to recovery a bad LIF, but here i need to know what exactly caused that unbootable situation.and now the problem of the IO chassis has been sealed, and if there is problem with LIF, i could make a alternate boot disk mirrored again.
what i want to know is in such circumstance, how can i figure out whether there was problem with the alternate boot disk, and could i boot the partition in such enviroment?

thanks

[Torsten.]

Hi,

I don't know your SD model, but I guess the failed IO chassis contains the core I/O card.

Can you confirm this?

Without this card the partition cannot boot.

[stephen peng]

hi Torsten,
there are two IO chassises, two CELLs configured in that partition, and both IO chassises have the same configuration, that means, both have core IO card. when i tried to boot the partition in which one IO chassis failed, using the alternate boot disk, connecting to the good IO chassis, it did not ask me to go in ISL or not, it directly told: bad LIF magic.
after the failed IO chassis changed, i had to reboot the partition asap, so i did not test the alternate boot disk. and now i just want to figure out in which part the problem raised?
and i also want to know, in such circumstance, if i power off the Cell that connected to the failed IO chassis, would that partition bootable?

thank you for your reply!


stephen

[Torsten.]

OK stephen, now check if the configuration of the disk is good using tools like

lifls
lvlnboot -v
setboot

lifls should list you at least:

>lifls -v /dev/rdsk/c0t6d0
HPUX 1
ISL 1
AUTO 1
LABEL 1

[Phil uk]

Has the Alternate disk ever been replaced in the past due to a hardware failure?
If so, it may be the case that the necessary "mkboot" commands weren't used (or used incorrectly) and subsequently the disk hasn't been tested since by booting the machine from it at a convenient time.