topic Re: telnet is core dumping (Signal 11) in Operating System - HP-UX

Telnet is core dumping

manju_102 — Fri, 12 Oct 2012 07:23:38 GMT

Hi,

My application is using telnet to connect to other NE. After establishing a telnet session, my application will send TL1 command ACT-USER, once telnet receives this command telnet is core dumping.

>gdb /usr/bin/telnet core

HP gdb 6.1 for HP Itanium (32 or 64 bit) and target HP-UX 11iv2 and 11iv3.

Copyright 1986 - 2009 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 6.1 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.
..(no debugging symbols found)...
Core was generated by `telnet'.
Program terminated with signal 11, Segmentation fault.
SEGV_MAPERR - Address not mapped to object
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
(no debugging symbols found)...(no debugging symbols found)...
#0 0x401db30:1 in tn+0x5f1 ()
(gdb)

Basically this is random issue. Could you please help me to resolve this issue ?

Compiler version: HP C/aC++ for Integrity Servers B3910B A.06.25.02 [Nov 25 2010]

gdb version: HP gdb 6.1 for HP Itanium (32 or 64 bit) and target HP-UX 11iv2 and 11iv3.

Thank you

Re: telnet is core dumping (Signal 11)

Dennis Handly — Fri, 12 Oct 2012 09:57:48 GMT

>Basically this is random issue. Could you please help me to resolve this issue?

If you have support, why not contact the Response Center?

Also, do you have the latest telnet patches, if any?

>#0 0x401db30:1 in tn+0x5f1 ()

Here you should ask for a back trace and register dump:

(gdb) bt

(gdb) info reg

(gdb) disas $pc-16*12 $pc+16*4

Re: telnet is core dumping (Signal 11)

manju_102 — Fri, 12 Oct 2012 10:19:50 GMT

Hi,

Find the attached for

(gdb) bt

(gdb) info reg

(gdb) disas $pc-16*12 $pc+16*4

Thank you

Manju

Re: telnet is core dumping (Signal 11)

Dennis Handly — Fri, 12 Oct 2012 12:04:36 GMT

>(gdb) bt

>#0 0x401db30:1 in tn+0x5f1 ()

If you can't trace the stack, something is terribly wrong.

>gr1: 0xc000000000000c9c

The gp value shouldn't start with 0xc.

Re: telnet is core dumping (Signal 11)

manju_102 — Fri, 12 Oct 2012 12:30:25 GMT

Hi Dennis,

Thank you so much for you support.

>The gp value shouldn't start with 0xc.

What is gr1, is this register ?

So now how to solve the problem. I ran strings -a command on core file will this output helps ?

Or is there any other way to solve this issue ?

Re: telnet is core dumping (Signal 11)

Dennis Handly — Fri, 12 Oct 2012 18:59:09 GMT

>What is gr1, is this register?

gp/gr1 is the global data pointer for the current load module.

>So now how to solve the problem?

It could be a hardware problem.

Every time it fails, do you end up in the same location with the same register values?

How often does it fail? How many commands are you sending to telnet before it does?

>I ran strings -a command on core file will this output helps?

Not usually. You might be able to see the last telnet command and output but only you would know that.

>Or is there any other way to solve this issue?

Probably not without access to a packcore and most likely you need to run it live in a debugger.

Re: telnet is core dumping (Signal 11)

manju_102 — Mon, 15 Oct 2012 09:12:41 GMT

> Every time it fails, do you end up in the same location with the same register values?

Yes, 3 times core happened, in all the core files register values are same.

>How often does it fail? How many commands are you sending to telnet before it does?

in this year 3 times core happened(Feb, May, Sept). I cant find because its in production.

In 3 core files no debugging symbols found.

Re: telnet is core dumping (Signal 11)

Dennis Handly — Mon, 15 Oct 2012 10:59:07 GMT

>Yes, 3 times core happened, in all the core files register values are same.

That would seem to point to a software bug.

Re: telnet is core dumping (Signal 11)

manju_102 — Thu, 22 Nov 2012 13:03:02 GMT

Hi,

My application binary is using these are the dynamic dependencies of executable files

bin> ldd TL1ad

TL1ad:
        libunalign.so.1 =>      /usr/lib/hpux64/libunalign.so.1
        libuca.so.1 => /usr/lib/hpux64/libuca.so.1
        libnsl.so.1 => /usr/lib/hpux64/libnsl.so.1
        libstd_v2.so.1 =>       /usr/lib/hpux64/libstd_v2.so.1
        libCsup.so.1 => /usr/lib/hpux64/libCsup.so.1
        libm.so.1 =>    /usr/lib/hpux64/libm.so.1
        libunwind.so.1 =>       /usr/lib/hpux64/libunwind.so.1
        libc.so.1 =>    /usr/lib/hpux64/libc.so.1
        libdl.so.1 =>   /usr/lib/hpux64/libdl.so.1
        libxti.so.1 => /usr/lib/hpux64/libxti.so.1

And my application uses telnet to connet the remote machine and telnet dynamic dependencies are

ldd /usr/bin/telnet

/usr/bin/telnet:
        libdl.so.1 =>   /usr/lib/hpux32/libdl.so.1
        libkrb5.so.1 => /usr/lib/hpux32/libkrb5.so.1
        libc.so.1 =>    /usr/lib/hpux32/libc.so.1
        libk5crypto.so.1 =>     /usr/lib/hpux32/libk5crypto.so.1
        libcom_err.so.1 =>      /usr/lib/hpux32/libcom_err.so.1

Here we can observe TL1ad uses 64 bit libraries and telnet is using 32 bit libraries .....telnet is dumping core because of this ?

Thank you

Manju

Re: telnet is core dumping (Signal 11)

Dennis Handly — Thu, 22 Nov 2012 13:19:33 GMT

>telnet is dumping core because of this?

No, these are separate processes.

Re: telnet is core dumping (Signal 11)

manju_102 — Thu, 22 Nov 2012 13:24:26 GMT

Hi,

If I get packcore from the production box will it help to analyze the issue ?

Thank you

Manju

Re: telnet is core dumping (Signal 11)

Dennis Handly — Thu, 22 Nov 2012 13:34:24 GMT

>If I get packcore from the production box will it help to analyze the issue?

Well, it is better than what you have now and at least you have a copy that you can analyze on another system.

Re: telnet is core dumping (Signal 11)

manju_102 — Wed, 28 Nov 2012 14:41:07 GMT

I got packcore file.

I ran unpackcore, its also showing same result.

>>gdb
HP gdb 6.1 for HP Itanium (32 or 64 bit) and target HP-UX 11iv2 and 11iv3.
Copyright 1986 - 2009 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 6.1 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.

(gdb) unpackcore
Unpackcore of packcore.tar is done!.
Do you want to remove packcore.tar ?(y or n) n
(no debugging symbols found)...
Core was generated by `telnet'.
Program terminated with signal 11, Segmentation fault.
SEGV_MAPERR - Address not mapped to object
#0 0x401db30:1 in tn+0x5f1 ()
(gdb) bt
#0 0x401db30:1 in tn+0x5f1 ()

Is there any other commands I can run and get some hint ?

Re: telnet is core dumping (Signal 11)

Dennis Handly — Wed, 28 Nov 2012 16:10:22 GMT

>Is there any other commands I can run and get some hint?

This is what you had before. It indicates that gp has been blasted. This was saved in r34 and that should have been valid on the last call:

0x401daf0:2 <tn+0x5b2>: br.call.sptk.many rp=b7;;

I'd have to look at my version to see what was being called.

What does "info shared" show?

And "elfdump -S -o your-core-file"?

I suppose you could have a stack overflow and destroy the RSE location of r34 but the current value shows 43 MB free.

What is the kernel parm maxssiz?

Re: telnet is core dumping (Signal 11)

Dennis Handly — Thu, 29 Nov 2012 07:08:30 GMT

>This was saved in r34 and that should have been valid on the last call:

>0x401daf0:2 <tn+0x5b2>: br.call.sptk.many rp=b7;;

In thinking what could be messing up local variables, I thought of setjmp/longjmp. And sure enough the function being
called is setjmp!

So the likely problem is that someone is calling longjmp on a jmp_buf that is out of scope. The value of r36 should be the return address from tn and it is a global data address and that's why bt fails.

I don't see any obvious fixes for your problem in PHNE_42509.

Re: telnet is core dumping (Signal 11)

manju_102 — Thu, 29 Nov 2012 13:19:45 GMT

> What does "info shared" show?

(gdb) info shared
Shared Object Libraries
tstart tend dstart dend gp
packcore/modules//dld.so
0x60000000c001c000 0x60000000c00d3bf0 0x200000007d7f5000 0x200000007d7f9cd8 0x200000007d7f7880
packcore/modules//libdl.so.1
0x60000000c09c0000 0x60000000c09c3ed0 0x200000007d7fe000 0x200000007d7fe320 0x200000007d7fe0c8
packcore/modules//libkrb5.so.1
0x60000000c3958000 0x60000000c3a654f0 0x200000007d7e8000 0x200000007d7eb6a0 0x200000007d7ea7e8
packcore/modules//libc.so.1
0x60000000c06e4000 0x60000000c09bd5a0 0x200000007d7d4000 0x200000007d7e7aa8 0x200000007d7dc5d0
packcore/modules//libk5crypto.so.1
0x60000000c3a68000 0x60000000c3ab3f70 0x200000007d7f0000 0x200000007d7f1c30 0x200000007d7f14c0
packcore/modules//libcom_err.so.1
0x60000000c2f90000 0x60000000c2f93f10 0x200000007d7f4000 0x200000007d7f4160 0x200000007d7f4090
Total of 6 shared libraries.

> And "elfdump -S -o your-core-file"?

manju>>elfdump -S -o core17-03-35-1

core17-03-35-1:

*** Program Header ***

Type Offset Vaddr FSize Memsz

CoreVer 00000314 00000000 00000004 00000004
CoreKern 00000318 00000000 00000008 00000008
CoreUTS 00000320 00000000 00000808 00000808
CoreComm 00000b28 00000000 00000006 00000006
CoreProc 00000b30 00000000 0000be00 0000be00
CoreLoad 0000c930 40010000 00010000 00010000
CoreMMF 0001c930 7d7cf000 00001000 00001000
CoreMMF 0001d930 7d7d0000 00004000 00004000
CoreMMF 00021930 7d7d4000 00008000 00008000
CoreMMF 00029930 7d7dc000 0000c000 0000c000
CoreMMF 00035930 7d7e8000 00004000 00004000
CoreMMF 00039930 7d7ec000 00004000 00004000
CoreMMF 0003d930 7d7f0000 00002000 00002000
CoreMMF 0003f930 7d7f2000 00002000 00002000
CoreMMF 00041930 7d7f4000 00001000 00001000
CoreMMF 00042930 7d7f5000 00003000 00003000
CoreMMF 00045930 7d7f8000 00002000 00002000
CoreMMF 00047930 7d7fa000 00002000 00002000
CoreMMF 00049930 7d7fc000 00001000 00001000
CoreMMF 0004a930 7d7fd000 00001000 00001000
CoreMMF 0004b930 7d7fe000 00001000 00001000
CoreStck 0004c930 7d7ff000 00001000 00001000
CoreStck 0004d930 7fff8000 00008000 00008000

> What is the kernel parm maxssiz?

homes/manju> /usr/sbin/sysdef | grep maxssiz
maxssiz                   20480          -          0-655360     Pages   -
maxssiz_64bit              8192          -        256-1048576    Pages   -

homes/manju> /usr/sbin/sysdef | grep maxdsiz
maxdsiz                  524288          -          0-655360     Pages   -
maxdsiz_64bit           1572864          -        256-1048576    Pages   -

homes/manju> /usr/sbin/sysdef | grep maxtsiz
maxtsiz                   32768          -          0-655360     Pages   -
maxtsiz_64bit            262144          -        256-1048576    Pages   -

Re: telnet is core dumping (Signal 11)

Dennis Handly — Thu, 29 Nov 2012 18:28:36 GMT

>(gdb) info shared

I didn't really need this info after I figured out the there is a software problem with setjmp/longjmp handling in telnet.

So try installing PHNE_42509. If that doesn't fix it, you'll need to contact the Response Center/Support Center and file a bug:

http://www.hp.com/go/hpsc

>/usr/sbin/sysdef | grep maxssiz

(Instead of using the (TO BE OBSOLETED) sysdef(1m), you should be using kctune(1m).)