rp3410 panic: data page fault error

[enrico.nic]

Hi

This morning my system (an rp3410 with HP-UX 11.11) rebooted unexpectedly, reporting a data page fault error.
This is the content of the file /var/adm/crash/crash.1/INDEX

comment savecrash crash dump INDEX file
version 2
hostname gissi
modelname 9000/800/rp3410
panic Data page fault
dumptime 1200557770 Thu Jan 17 09:16:10 MET 2008
savetime 1200558330 Thu Jan 17 09:25:30 MET 2008
release @(#) $Revision: vmunix: vw: -proj selectors: CUPI80_BL2000_1108 -c 'Vw for PI80_BL2000_1108 build' -- cupi80_bl2000_1108 'CUPI80_BL2000_1108' Wed Nov 8 19:24:56 PST 2000 $
memsize 5366611968
chunksize 268435456
module /stand/vmunix vmunix 19168752 3734139179
module /stand/dlkm/mod.d/pfil pfil 176528 1398022812
module /stand/dlkm/mod.d/rng rng 68688 3212794757
module /stand/dlkm/mod.d/ipf ipf 456864 4106557605
module /stand/dlkm/mod.d/gvid_info gvid_info 22624 3984583571
image image.1.1 0x0000000000000000 0x000000000fffb000 0x0000000000000000 0x0
000000000010ba7 3377961417
image image.1.2 0x0000000000000000 0x000000000c891000 0x0000000000010ba8 0x0
00000000003ffff 4109058479
image image.2.1 0x0000000000000000 0x000000000461b000 0x0000000000100000 0x0
00000000013fdff 2956101342
image image.3.1 0x0000000000000000 0x000000000fff1000 0x0000000004040000 0x0
000000004087a1f 1766864377
image image.3.2 0x0000000000000000 0x000000000fff1000 0x0000000004087a20 0x0
0000000040c374f 1965833465
image image.3.3 0x0000000000000000 0x0000000000910000 0x00000000040c3750 0x0
0000000040fffff 4167020378

Can anybody please give me a hint where to look for to solve the problem ?

Thank you in advance

Enrico

[Don Morris_1]

Not without the stack trace.

Run q4 in the /var/adm/crash/crash.1 directory (it may request processing the vmunix first) and report:

trace event 0

and

examine &msgbuf+0x8 using s

Alternately, contact Support for crashinfo [or get an older version that should be fine for 11.11 from thread:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1089090&admit=109447626+1200584198486+28353475
] and report the result of that.

[Bryan Rooney]

I'm attaching the 3.9 PA-RISC version of the program (unsupported HP Support tool called crashinfo is not on most systems).

Place crashinfo in the same dir as the crash.1 directory. The output should contain the backtrace HP Support will want to see.

Run:

# cd /var/adm/crash/crash.1
# ./crashinfo -c > output.txt

The progran may or may not work with you existing HP-UX libraries.

[Mridul Shrivastava]

Data page fault is caused by software and cause of the panic in these cases mostly is software issue.

Do you have output of crashinfo or could you please post the output of crashinfo.. since the information posted is not sufficient to confirm the cause of the panic.

Most of the cases it turn out to be some driver or patch issue as well.

If nothing helps then I would suggest you to log a call with HP if this is a critical server then do that as the first thing.

[Mridul Shrivastava]

If you can post the stack trace the may be someone can help you out.
Have a look at the similar threads:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1170105&admit=109447626+1200586063262+28353475

http://forums12.itrc.hp.com/service/forums/questionanswer.do?threadId=1098272&admit=109447627+1200586168220+28353475

[enrico.nic]

Hi

Thank you for the replies: sorry for the delay, I didn't have crashinfo installed, and I was unable to extract it from Bryan Rooney's attachment.
Finally I've found the utility on the hprc.external.hp.com FTP site, and here is the output from a
# ./crashinfo -c > output.txt
command.

Thank you for any help you can give me: this is a critical server, and since January,1 I am without a support contract (still waiting for an HP proposal of renewal).

Enrico

[Don Morris_1 · Accepted solution]

Looks like QXCR1000590932 / JAGag46672:

Symptom:
System may panic with following stack trace:
ill6_lookup_on_name_real+0x94
ill6_lookup_on_name+0x14
if_lookup_on_name+0x54
pfil:qif_update+0x48
pfil:pfilwput_ioctl+0xbc
pfil:pfilstrmodwput+0x170

Defect Description:
A system installed with IPFilter (B9901AA) version
A.03.05.12 or earlier and NOT installed with IPv6 (IPV6AA)
may panic after installing ARPA Transport patch PHNE_35351.

You show the stack, you show having the patch.

The fix isn't yet generally available, unfortunately. I know you said your support contract renewal was in processing limbo - but if you can contact support and request the fix for this (give them the issue number above) they can do such limited releases.

[Mridul Shrivastava]

I am not an expert in analysing the crashinfo o/ps.. its been days now since I analysed the one. However looking at this o/p I feel this is something related to igelan interface available on your system

igelan0: INITIALIZING HP PCI 1000Base-T Core at hardware path 0/1/2/0

Can you check for network patches.. I am also trying to run the patch assessment using the patc information available from crashinfo o/p.

I think this should get resolve after installting igelan latest patch since igelan has lots of bugs.

However it is a critical system so I still recommend you to take a opinion from HP. If they delayed the contract renewal then they should provide you support at no extra cost.

[Mridul Shrivastava]

Try running the patch assessment , for details refer the following:

http://www11.itrc.hp.com/service/patch/wrap.do?pageKey=patch.html.howToPerformAssessment

[Tim Nelson]

Just put a call into HP. Tell them the renewal is in process.

If you do not renew they will just bill you for the call. If you renew no problem. Have done this many times in the past.

Boils down to how critical this is.

[Prashanth.D.S]

Hi Enrico,

Stack matches exactly with a know/identified issue on a system installed with PHNE_35351 & IPF Filter.

This is due to a mismatch between PHNE_35351 and old versions of IPFilter (before A.03.05.14).

PHNE_35351 introduced some
new code in function if_lookup_on_name() that handles differently for
ipv6 and ip4. This function is called by IPFilter's qif_update()
function. However if the IPFilter is very old on the system then it doesn't pass the appropriate data to if_lookup_on_name(), so if_lookup_on_name() thinks the ip version is 6, thus it does ipv6 related processing and causes panic.


You should remove IPFilter if you don't use it, and install A.03.05.14 if you need to use IPFilter

Best Regards,
Prashanth

[enrico.nic]

Hi

Thank you for telling me about the problem JAGag46672, it seems that with a newer release of IPFilter (just installed) the problems with the ARPA transport patch PHNE_35351 should go away. As Don Morris and Prashanth.D.S pointed out, this problem is exactly reported in the patch documentation.
Since the panic happened just once, I will wait until the contract renewal before opening the call.

For anybody interested in crashinfo, the following links worked for me:

for 64-bit (PA-RISC or IA64):
ftp://collect:scripts@hprc.external.hp.com/memory.64bit.tar.Z

for 32-bit: ftp://collect:scripts@hprc.external.hp.com/memory.32bit.tar.Z

README file
ftp://collect:scripts@hprc.external.hp.com/memory.README.txt

Thank you all for your time, skills and rapidity

Enrico

[Don Morris_1]

FYI, the ARPA Transport patch which contains the fix posted this morning. PHNE_36125.