system panic igniting from make_tape_recovery.

[clive musgrove_2]

Doing a DR recovery test of an 11.11 RP2470.
System is patched bundled to June 2003 and has Ignite B.4.4.30.
Made 2 DLT IV tapes (to a local surestore)using make_tape_recovery -A, one with the standard mnr_essentials and the other including a number of other paths in mnr_essentials.
The MTR completes succesfully (with normal warnings) and the LIF is written.
When I boot from the either tape the system panics:

ISL booting hpux (;0):INSTALL
Boot
: tape(0/4/0/1.1.0.0.0.0.0;0):WINSTALL
11538432 + 1941592 + 2610520 start 0x201468
alloc_pdc_pages: Relocating PDC from 0xf0f0000000 to 0x3fb00000.

10 minutes later............
************* SYSTEM ALERT **************
SYSTEM NAME: uninitialized
DATE: 10/07/2003 TIME: 13:46:47
ALERT LEVEL: 12 = Software failure

REASON FOR ALERT
SOURCE: 1 = processor
SOURCE DETAIL: 1 = processor general SOURCE ID: 0
PROBLEM DETAIL: 0 = no problem detail

LEDs: RUN ATTENTION FAULT REMOTE POWER
FLASH FLASH OFF ON ON
LED State: Running non-OS code. Non-critical error detected.
Check Chassis and Console Logs for error messages.

0xF8E000C01100B800 00000000 0000B800 - type 31 = legacy PA HEX chassis-code
0x58E008C01100B800 00006709 070D2E2F - type 11 = Timestamp 10/07/2003 13:46:47


The GSP shows 2 "ALERT LEVEL: 3 = System blocked waiting for operator input" messages followed by Log Entry # 2 :
SYSTEM NAME: uninitialized
DATE: 10/07/2003 TIME: 13:46:47
ALERT LEVEL: 12 = Software failure

SOURCE: 1 = processor
SOURCE DETAIL: 1 = processor general SOURCE ID: 0
PROBLEM DETAIL: 0 = no problem detail

CALLER ACTIVITY: B = system panic STATUS: 0
CALLER SUBACTIVITY: 00 = implementation dependent
REPORTING ENTITY TYPE: E = HP-UX REPORTING ENTITY ID: 00

0xA0E000C01100B000 00000000 000005E9 type 20 = major change in system state
0x58E008C01100B000 00006709 070D2E2F type 11 = Timestamp 10/07/2003 13:46:47
Type CR for next entry, - CR for previous entry, Q CR to quit.



Log Entry # 3 :
SYSTEM NAME: uninitialized
DATE: 10/07/2003 TIME: 13:46:47
ALERT LEVEL: 12 = Software failure

SOURCE: 1 = processor
SOURCE DETAIL: 1 = processor general SOURCE ID: 0
PROBLEM DETAIL: 0 = no problem detail

CALLER ACTIVITY: B = system panic STATUS: 0
CALLER SUBACTIVITY: 80 = implementation dependent
REPORTING ENTITY TYPE: E = HP-UX REPORTING ENTITY ID: 00

0xF8E000C01100B800 00000000 0000B800 type 31 = legacy PA HEX chassis-code
0x58E008C01100B800 00006709 070D2E2F type 11 = Timestamp 10/07/2003 13:46:47
Type CR for next entry, - CR for previous entry, Q CR to quit.

Anyone got any ideas ?

[V.Tamilvanan]

Hi,
Try by using the following command which I use normally to take Ignite-UX backup and even I have recovered so many times.

#make_tape_recovery -x inc_entire=vg00 -Iv -a /dev/rmt/0mn

[Hari Kumar]

The suggested method to trigger Ignite :
#make_tape_recovery -a /dev/rmt/?mn -I -v -x inc_entire=vg00

We can verify the Tape with following steps ---
1. Check the log generated during the Ignite backup:(best place to see first)
/var/opt/ignite/logs/makrec.logxx
2.
#dd if=/dev/rmt/0mn of=/whereis/lifarea bs=2k
Now run lifls and lifcp commands on that file to verify the area
#lifls /whereis/lifarea

3. The second image on the tape is the tar archive of the OS and
possibly other files (depending on the options to make_recovery).
We can Skip over the first image with mt command using the appropriate tape device file;
mt -f /dev/rmt/0mn rew
mt -f /dev/rmt/0mn fsf 1
We can Verify the contents of the tar image with the tar command using the appropriate device file;
tar tvf /dev/rmt/0m
4. we are also given 'check_recovery' with ignite, probably it requires -C flag at make_recovery.

HTH

[Hari Kumar]

One more utility shipped with Ignite is
copy_boot_tape
Checking boot volume on tape (LIF header)
#/opt/ignite/bin/copy_boot_tape -u /dev/rmt/0mn -b -d /tmp
#lifls â l /tmp/bootim

[Dietmar Konermann]

Clive,

obviously your Ignite WINSTALL kernel paniced the system. Usually this happens when using "older" Ignite versions with "newer" systems. But this seems not to be the problem here.

Up to now you only posted the hardware log entries generated by GSP. Please try to get the console output that the kernel printed while panicing. Use the GSP's CL command and try to find the "Panic string".

Best regards...
Dietmar.

[clive musgrove_2]

Hi Tamil, we are aiming for a complete system restore tape and want to recover multiple mount points across a number of VG's - we have tried make_tape_recovery -x on other systems before but had problems. We have used the mnr_essentials before with more success.
Vamsi, I had verified the make_tape_recovery logs, there isn't a /var/opt/ignite/logs/makrec.log (is this not for make_recovery only ?).
A lifls on the copied li file shows :
ISL AUTO INDEX CONFIG HPUX
FWWKAR6 FWWKAR7 FWWKAR8 INSTALL INSTALLFS
VINSTALLFS WINSTALLFS VINSTALL WINSTALL INSTCMDS
SYSCMDS SCRIPTS VERSION
and a copy_boot_tape shows :
46073+1 records in
46073+1 records out
Boot file contents.
volume ISL10 data size 804676 directory size 3 03/10/06 16:33:50
filename type start size implement created
===============================================================
ISL -12800 16 306 0 03/10/06 16:33:50
AUTO -12289 328 1 0 03/10/06 16:33:50
INDEX BIN 336 1 0 03/10/06 16:33:50
CONFIG BIN 344 145 0 03/10/06 16:33:50
HPUX -12928 496 848 0 03/10/06 16:33:50
FWWKAR6 BIN 1344 1 0 03/10/06 16:33:50
FWWKAR7 BIN 1352 1 0 03/10/06 16:33:50
FWWKAR8 BIN 1360 1 0 03/10/06 16:33:50
INSTALL -12290 1368 67604 0 03/10/06 16:33:55
INSTALLFS -12290 68976 35840 0 03/10/06 16:33:57
VINSTALLFS -12290 68976 35840 0 03/10/06 16:33:57
WINSTALLFS -12290 68976 35840 0 03/10/06 16:33:57
VINSTALL -12290 104816 73206 0 03/10/06 16:34:01
WINSTALL -12290 178024 83036 0 03/10/06 16:34:03
INSTCMDS BIN 261064 29885 0 03/10/06 16:34:04
SYSCMDS BIN 290952 77577 0 03/10/06 16:34:05
SCRIPTS BIN 368536 44 0 03/10/06 16:34:05
VERSION BIN 368584 1 0 03/10/06 16:34:05
I thought the check_recovery was only for make_recovery ?
The tar image on the tape is intact - I have verified the number of files.
Dietmar,
Unfortunately the server doesn't have a remote console, and I am not on site for another week. I will update the CL details then.
Many thanks so the help so far....

[Steven E. Protter]

You can accomplish your goal.

On some older systems, I got panics on Ignite restores because of the fiber cards. To get them to boot, I had to disconnect the cards, boot, and then reconnect them after the kernel loaded.

Pretty old hardware though.

SEP

[Marco Santerre]

As was mentionned already, an older version of Ignite on the source server could be the problem, though I have my suspicions set on your destination server having new hardware that your source server would not be able to handled at this time.

What I mean is, chances are you're missing a couple of drivers or patches to enabled some newer hardware that is on your destination server. Load the newest Hardware Enablement patches on your source server, and your Ignite should be ready to go.

[clive musgrove_2]

Hi Steven,
This server is all scsi, with a dual port scsi connected to the tape drive and one end of a mirrored disk enclosure, and the other single port scsi card connected to the other end of the mirrored disk enclosure.
The only cards I could remove are the single port scsi and the NIC's, but they are all brand new.
Marco
The make_tape_recovery was run on the paniced machine.
When doing the recovery the server panics, and it automatically reboots successfully from the disks.
Thanks

[clive musgrove_2]

I got someone to go through the GSP CL logs and they didn't identify any error.
I did notice on dmesg an error:
SCSI: Ultra160 Controller at 0/4/0/1: Warning: Data transfer rate stepped down f
or target 1. Now operating at 20 MB/s (Fast Wide). Possible causes are improper
termination, improper cabling, or malfunctioning hardware.
I will look into this since this is the controller handling the tape drive.