rx2660 panic reboot

[Girdharilal]

HP-UX 11iv2 running application and it reboot after panic. We have check the logs but not showing any hardware error.

Please find virtual memory status.

faults
in sy cs
1913 194298 651

# vmstat -n 5
VM
memory page faults
avm free re at pi po fr de sr in sy cs
740143 1427402 9490 164 1 0 0 0 5 1913 194298 651
CPU
cpu procs
us sy id r b w
8 7 86 1 0 0
8 7 86
8 7 86
8 7 86
622559 1427091 2143 32 0 0 0 0 0 1826 189794 593
6 4 90 1 1 0
6 3 90
10 7 84
5 4 91
622559 1416452 3301 35 13 0 0 0 0 1815 172038 551
23 5 72 1 1 0
13 7 81
14 3 83
8 7 84
715738 1426568 2127 34 3 0 0 0 0 1872 265254 578
9 10 81 2 0 0
7 6 86
6 5 89
7 6 87
â ¥#

â ¥# vmstat
procs memory page
faults cpu
r b w avm free re at pi po fr de sr in
sy cs us sy id
1 1 0 768227 1426493 9490 164 1 0 0 0 5 191
194298 651 8 7 86
#

[Sunny123_1]

Hi Girdharilal

Did you check any crash or ts file genrated

ll /var/adm/crash/

ll /var/tombstone/ts99

Regards
Sunny

[mobidyc]

Hello,

Do you have any /var/tombstones/ts* file ?

also, If you have /var/adm/crash/crash.X directory, go into it and run:
crashinfo >/tmp/crash.log

it's the best way to analyse a panic problem.

BR,
Cedrick Gaillard

[Patrick Wallek]

Virtual memory is not really relevant here, especially after the fact.

Check the following for information on the crash:

/var/adm/syslog/OLDsyslog.log
/etc/shutdownlog
/var/adm/crash --- If there is a crash dump here, it is best to get HP to analyze it.

[Girdharilal]

I have posted crashlogs earlier. I dont see any hardware error on panic and infact , it generate similar core dump everytime its reboot.

Please find previous dump from MP LAN.

Console Login: Calling function e00000000103f4e0 for Shutdown State 8 type 0x2

Stored message buffer up to panic:
a20175
4- 7 20000000_73426960 00000000_00000000 00000000_00000000 00000000_00000000
8-11 e0000001_a378ed6c 00000000_000005b4 00000000_00000001 e0000001_b93d5d88
12-15 e0000001_0173fb40 9fffffff_7f5c8000 e0000001_ba7db0c0 e0000001_a378eb74
16-19 e0000001_a378eb30 e0000001_0173fb40 e0000001_00abae78 e0000001_bd55b068
20-23 00000000_00000083 00000000_00000007 e0000001_bd55b140 e0000001_01740000
24-27 00000000_0000bfc6 e0000000_f00000d8 e0000001_a378eb74 e0000001_a260b7c8
28-31 e0000000_f00000d8 00000000_0000bfc6 e0000001_a260b7d8 e0000000_f0000000


br_0-3 e0000000_00609340 e0000000_005be7b0 00000000_00000000 00000000_00000000
br_4-7 00000000_00000000 00000000_00000000 e0000000_005d9d40 e0000000_00693a50

3 3 2 22 2 1 0 00 0
pr bits = --------------------------7----2---8---43-1---7-------9---54---0
pr value = 00000021_11a20231

k0 00000000_0000045b rsc 00000000_00000013 fpsr 0009804c_0270033f
k1 00000000_00042a28 bsp e0000001_0172c8e0 unat ffffffff_ffffffff
k2 00000000_00000000 bspstore e0000001_0172c6d0 lc 00000000_00000000
k3 00000000_00000000 bsp_base 00000000_00000000
ppdp e0000001_01740000 dirty 00000000_00000210 csd 00000000_00000000
ktp e0000001_a37e4040 rnat 00000000_00000000 ssd 00000000_00000000
ksp 9fffffff_7f5c8883 ccv 00000000_5e5c0680
sv e0000001_08003000 pfs 00000000_00001cc1 ec 00

iip e0000000_00609380/0 ifs 80000000_00001cc1 [i]psr 00001010_086ae01a
iipa e0000000_00609370 tpr 00000000_000000c0 isr 00000002_00000010

ibe sddiimic rtldsdpsddd p i mmaub

andrisdadtcspl tbpbiipphlt kic hlcpe
psr bits = ------------------01000000010000----10000110101-111-------01101-

e snirsn
deioirsparwx < code >
isr bits = --------------------000000000010--------000000000000000000010000

arg0=e0000000_00609380 (cr.iip)
arg1=00000000_00000000 (unknown)

Dirty Registers:
loc0 : e0000001_b93d5bc0 e0000001_b9713800 00000000_00000d9f e0000000_005be990
loc4 : 00000000_00000000 00000021_11a20133 e0000001_0173fb60 e0000001_0173fb10
loc8 : e0000001_b9713965 e0000001_00804f70 e0000001_00c50890 e0000001_a378ec74
locc : e0000001_a378ec40 e0000001_a378eb64 e0000001_a378ec68 e0000001_a378ec6c
loc10: e0000001_a378eb68 e0000001_a378ec8c e0000001_a378eb58 e0000001_a378eb60
loc14: 00000000_00000000 00000000_00000000 e0000001_b9713964 e0000001_a378eb6c
loc18: e0000001_b9713967 e0000001_b9713966 e0000001_a378ee7c e0000001_b9713958
loc1c: e0000001_a378ebf6 00000000_0000ffff e0000001_a378ec0c e0000001_a378ee78
loc20: e0000001_a378ec08 e0000001_a378ebb8 00000000_00000112 e0000001_a378ee90


loc24: 00000000_5a3969f0 e0000001_a378ec70 e0000001_a378eb28 e0000001_a378ecc0
loc28: e0000001_0173fb70 00000000_00000000 00000000_000000ff e0000000_f0000058
loc2c: e0000001_b9713944 00000000_00008000 00000000_ad915368 e0000001_b9713820
loc30: e0000001_b9713818 00000000_ad915369 e0000001_a378eba8 e0000001_006e3548
loc34: 00000000_00000000 ffffffff_ffffff7f e0000001_a378ecc4 00000000_000000cc
loc38: 00000000_00001125
out0 : e0000001_a378eb28 e0000001_b9713944 e0000001_b9713958 e0000001_a378ed10
out4 : 00000000_00000000 00000000_000005b4 00000000_00000001 e0000001_b9713976

System Panic:

panic: Bad News!

Stack Trace:
IP Function Name
0xe000000000c8f4e0 bad_news+0x950
0xe000000000c8e500 bubbledown
0xe000000000609380 tcp_rput+0x22a0
0xe0000000005be990 ip_wput_ioctl+0x5d0
0xe0000000008507a0 soft_intr_handler+0x220


0xe00000000084ef30 external_interrupt+0x3b0
0xe000000000c8e500 bubbledown
0xe000000000693c00 syscall+0x1b0
End of Stack Trace

linkstamp: Thu Sep 06 18:48:23 EAT 2007
_release_version: @(#) $Revision: vmunix: B11.23_LR FLAVOR=perf Fri Aug 29
22:35:38 PDT 2003 $

NOT sync'ing disks (on the ICS) (0 buffers to flush):
0 buffers not flushed
144 buffers still dirty

*** A system crash has occurred. (See the above messages for details.)
*** The system is now preparing to dump physical memory to disk, for use
*** in debugging the crash.

*** Cannot dump with compression because there are too few processors
*** Dumping without compression
*** The dump will be a SELECTIVE dump: 2291 of 16365 megabytes.
*** To change this dump type, press any key within 10 seconds.

*** Proceeding with selective dump.



Primary Dump Header Location :
Device : /dev/rdsk/c1t0d0 offset: 1563508.
*** The dump may be aborted at any time by pressing ESC.
***********************************************************dsk/c1t0d0 )
* ROM Version : 01.05
* ROM Date : 11/06/2006
* BMC Version : 05.06
***********************************************************
0 0 0x0015B2 0x0000000020744198 boot time event
1 0 0x0000A4 0x0000000000000000 start memory configuration
2 0 0x001CBB 0x0000000000000000 System set to insecure mode
0 0 0x0015B2 0x0000000035764734 boot time event
1 0 0x000014 0x0000000000000000 CPU0 starting cell relocation
1 0 0x000009 0x0000000000000000 CPU0 launch EFI

[avizen9]

Hello Girdharilal

i will prefer /var/adm/crash/crash.x to Analysis, you may log call to hp and send crash dump to analysis, will get correct ans.

[doug hosking]

See page 23 of http://docs.hp.com/en/5991-5544/5991-5544.pdf

[Torsten.]

"I have posted crashlogs earlier"

Where? Did we missed them?

The firmware of this server is very, very old and needs to be updated (1.05 was initial version).

How about the OS? Patched up-to-date?

[Girdharilal]

Dear All,
thanks for your input. I have more lots about the server. Logs are attached and put your input.

I will try to update firmware and OS patch.

Regards
Girdhari