Application on 11.23 crashes under gdb

[FRESNEDA Christine]

We have an application compiled in debug mode that runs fine under 11.23.
But when we attach to it under gdb to set breakpoints it crashes:


gdb /home/dholt/vt_tl1_am -pid 15177
HP gdb 5.8 for HP Itanium (32 or 64 bit) and target HP-UX 11.2x.
Copyright 1986 - 2001 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 5.8 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.
..
Attaching to program: /home/dholt/vt_tl1_am, process 15177

warning: The shared libraries were not privately mapped; setting a
breakpoint in a shared library will not work until you rerun the program;
stepping over longjmp calls will not work as expected.
Use the following command to enable debugging of shared libraries.
chatr +dbg enable <executable>

warning: Load module /usr/opt/temip/lib/libclntsh.so.9.0 has been stripped.
Debugging information is not available.


warning: Load module /usr/opt/temip/lib/libwtc9.so has been stripped.
Debugging information is not available.

0xc00000000028ad30:0 in _select_sys+0x30 () from /usr/lib/hpux64/libc.so.1
(gdb) break vt_tl1am_comm.cxx:1095
Breakpoint 1 at 0x4000000000b3e0b0:0: file ../tl1_am/src/vt_tl1am_comm.cxx, line 1095 from /home/dholt/vt_tl1_am.
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault
si_code: 1 - SEGV_MAPERR - Address not mapped to object.
[Switching to thread 33 (system thread 1057098)]
MDequeElement<MVerb>::IsRoot (this=0x9fffffffbf1c26f0)
at /usr/opt/temip/tfc/include/tfc_sp_queue.hxx:201
201 MCC_T_Boolean IsRoot() const { return MCC_K_FALSE;}
Current language: auto; currently c++
(gdb) bt
0 MDequeElement<MVerb>::IsRoot (this=0x9fffffffbf1c26f0)
at /usr/opt/temip/tfc/include/tfc_sp_queue.hxx:201
1 0xc00000000be776a0:0 in MClass::OnMccCall () at ../tfc_sp_queue.hxx:91
2 0xc00000000be85640:0 in DispatchModuleSM () at ../tfc_fmk_self_mgt.cxx:103
3 0xc000000008e2f320:0 in im__dispatch_local () at ../mcc_dispatch.c:187
4 0xc000000008e790c0:0 in im__rpc_thread_main () at ../mcc_ipc_unix.c:4638
5 0xc000000008d957b0:0 in thread_shell () at ../mcc_fw_threads.c:488
6 0xc00000000030f320:0 in __pthread_bound_body+0x190 ()
from /usr/lib/hpux64/libpthread.so.1
(gdb) kill
Kill the program being debugged? (y or n) y
(gdb) quit

[Dennis Handly]

>But when we attach to it under gdb to set breakpoints it crashes:

Make sure you use chatr like it said:
chatr +dbg enable /home/dholt/vt_tl1_am

Does this abort consistently?

Have you tried debugging the signal 11 in MDequeElement::IsRoot, assuming it was your bug?

Can you print *this there?
Do you have a thread stack overflow?

[FRESNEDA Christine]

Yes it does core consistently.

>>Make sure you use chatr like it said:
>>chatr +dbg enable /home/dholt/vt_tl1_am
We are going to try this.


Have you tried debugging the signal 11 in MDequeElement::IsRoot, assuming it was your bug?
No but here is its definition:
MCC_T_Boolean IsRoot() const { return MCC_K_FALSE;}

>>Can you print *this there?
we will do and post the result.

>>Do you have a thread stack overflow?
How can we check for it?

Thank you,
Christine

[Dennis Handly]

>No but here is its definition:
MCC_T_Boolean IsRoot() const { return MCC_K_FALSE;}

If MCC_K_FALSE is a constant, I can't see how it fails unless "this" is bad?

>we will do and post the result.

You can also disassemble the function:
(gdb) disas
(gdb) info reg

>>Do you have a thread stack overflow?
>How can we check for it?

(gdb) frame 0
(gdb) p /x $save_sp = $sp
(gdb) frame 6
(gdb) p $save_sp - $sp

[FRESNEDA Christine]

Here are some answer:

$ kctune maxdsiz maxtsiz maxssiz

Tunable Value Expression Changes

maxdsiz 1073741824 Default Immed

maxssiz 134217728 134217728 Immed

maxtsiz 100663296 Default Immed





$ type gdb

gdb is hashed (/opt/langtools/bin/gdb)





$ /opt/langtools/bin/gdb --version

HP gdb 5.8 for HP Itanium (32 or 64 bit) and target HP-UX 11.2x.

Copyright 1986 - 2001 Free Software Foundation, Inc.

Hewlett-Packard Wildebeest 5.8 (based on GDB) is covered by the

GNU General Public License. Type "show copying" to see the conditions to

change it and/or distribute copies. Type "show warranty" for warranty/support.


$ aCC -V

aCC: HP C/aC++ B3910B A.06.14 [Feb 22 2007]

## Extract from a make:



source='tl1_am/src/vt_tl1am_module.cxx' object='tl1_am/src/vt_tl1_am-vt_tl1am_module.o' libtool=no \

DEPDIR=.deps depmode=hp2 /bin/sh ../build-aux/depcomp \

aCC +DD64 -Wl,+k -DHAVE_CONFIG_H -I. -I.. -DVZB_TL1_AM_CONF_LOCAL=\"/etc/opt/vzb/tl1_am\" -DVZB_TL1_AM_CONF_CLUSTER=\"/etc/opt/vzb/temip/tl1_am\" -DVZB_TL1_AM_LOG_LOCAL=\"/var/opt/vzb/log/tl1_am\" -DVZB_TL1_AM_LOG_CLUSTER=\"/var/opt/vzb/temip/log/tl1_am\" -DVZB_TL1_AM_TRACE_LOCAL=\"/var/opt/vzb/trace/tl1_am\" -DVZB_TL1_AM_TRACE_CLUSTER=\"/var/opt/vzb/temip/trace/tl1_am\" -DVZB_TL1_AM_STATE_LOCAL=\"/var/opt/vzb/tl1_am\" -DVZB_TL1_AM_STATE_CLUSTER=\"/var/opt/vzb/temip/tl1_am\" -DVZB_COMMON_STATE_LOCAL=\"/var/opt/vzb/common\" -DVZB_COMMON_STATE_CLUSTER=\"/var/opt/vzb/temip/common\" -DVISUAL_TEMIP -DYY_NO_UNPUT -I../tl1_am/src -I/usr/opt/temip/CMA/include -I/opt/vzb/dstring/include -DPTHREAD_COMPAT_MODE -D_HPUX_SOURCE -D_RWCONFIG_m -D__HPACC_USING_MULTIPLIES_IN_FUNCTIONAL -DDEBUG -I./model/msl -I../model/msl -I/usr/opt/temip/tfc/include -I/usr/opt/temip/mmtoolkit/include -I/usr/opt/temip/mmtoolkit/dev -I/usr/opt/temip/acloc/include -I/usr/opt/temip/ocs/include -mt +DD64 +z -AA!

-g -c -o tl1_am/src/vt_tl1_am-vt_tl1am_module.o `test -f 'tl1_am/src/vt_tl1am_module.cxx' || echo '../'`tl1_am/src/vt_tl1am_module.cxx



/bin/sh ./libtool --tag=CXX --mode=link aCC +DD64 -Wl,+k -mt +DD64 +z -AA -g -D_HPUX_SOURCE -mt -z +DD64 -DPTHREAD_COMPAT_MODE /usr/opt/temip/mmtoolkit/lib/mcc_mmshell_unix.o -L/usr/opt/temip/lib -R/usr/opt/temip/lib -ltfc_sp -ltfc_wp -ltfc_fmk -ltfc_acloc -ltfc_async -ltfc_event_filter -ltls77-m -lmcc_exec -lmcc_sys -R/usr/opt/temip/CMA/lib -L/usr/opt/temip/CMA/lib +noobjdebug -lcmi -o vt_tl1_am tl1_am/src/vt_tl1_am-vt_tl1am_sdhne.o tl1_am/src/vt_tl1_am-vt_tl1am_sdhne_children.o tl1_am/src/vt_tl1_am-vt_tl1am_session_manager.o tl1_am/src/vt_tl1_am-vt_tl1am_user_types.o tl1_am/src/vt_tl1_am-vt_tl1am.o tl1_am/src/vt_tl1_am-vt_tl1am_api.o tl1_am/src/vt_tl1_am-vt_tl1am_backend_test.o tl1_am/src/vt_tl1_am-vt_tl1am_comm.o tl1_am/src/vt_tl1_am-vt_tl1am_comm_link.o tl1_am/src/vt_tl1_am-vt_tl1am_common_test.o tl1_am/src/vt_tl1_am-vt_tl1am_mapper.o tl1_am/src/vt_tl1_am-vt_tl1am_mapper_parser.o tl1_am/src/vt_tl1_am-vt_tl1am_module.o tl1_am/src/vt_tl1_am-vt_tl1am_osi.o tl1_am/src/vt_!

tl1_am-vt_tl1am_parser_lex.o tl1_am/src/vt_tl1_am-vt_tl1am_parser_yacc.o /opt/vzb/dstring/lib/libDStringTeMIP.la



aCC +DD64 -Wl,+k -mt +DD64 +z -AA -g -D_HPUX_SOURCE -mt -z +DD64 -DPTHREAD_COMPAT_MODE /usr/opt/temip/mmtoolkit/lib/mcc_mmshell_unix.o +noobjdebug -o vt_tl1_am tl1_am/src/vt_tl1_am-vt_tl1am_sdhne.o tl1_am/src/vt_tl1_am-vt_tl1am_sdhne_children.o tl1_am/src/vt_tl1_am-vt_tl1am_session_manager.o tl1_am/src/vt_tl1_am-vt_tl1am_user_types.o tl1_am/src/vt_tl1_am-vt_tl1am.o tl1_am/src/vt_tl1_am-vt_tl1am_api.o tl1_am/src/vt_tl1_am-vt_tl1am_backend_test.o tl1_am/src/vt_tl1_am-vt_tl1am_comm.o tl1_am/src/vt_tl1_am-vt_tl1am_comm_link.o tl1_am/src/vt_tl1_am-vt_tl1am_common_test.o tl1_am/src/vt_tl1_am-vt_tl1am_mapper.o tl1_am/src/vt_tl1_am-vt_tl1am_mapper_parser.o tl1_am/src/vt_tl1_am-vt_tl1am_module.o tl1_am/src/vt_tl1_am-vt_tl1am_osi.o tl1_am/src/vt_tl1_am-vt_tl1am_parser_lex.o tl1_am/src/vt_tl1_am-vt_tl1am_parser_yacc.o -L/usr/opt/temip/lib -ltfc_sp -ltfc_wp -ltfc_fmk -ltfc_acloc -ltfc_async -ltfc_event_filter -ltls77-m -lmcc_exec -lmcc_sys -L/usr/opt/temip/CMA/lib -lcmi /opt/vzb/dstrin!

g/lib/libDStringTeMIP.so -L/usr/opt/temip/tfc/lib -L/usr/opt/temip/mmtoolkit/lib -Wl,+b -Wl,/opt/vzb/dstring/lib:/usr/opt/temip/lib:/usr/opt/temip/CMA/lib











bash-3.00$ file /home/dholt/vt_tl1_am

/home/dholt/vt_tl1_am: ELF-64 executable object file - IA64



chatr +dbg enable /home/dholt/vt_tl1_am

/home/dholt/vt_tl1_am:

current values:

64-bit ELF executable

shared library dynamic path search:

LD_LIBRARY_PATH enabled first

SHLIB_PATH enabled second

embedded path enabled third /opt/vzb/dstring/lib:/usr/opt/temip/lib:/usr/opt/temip/CMA/lib

shared library list:

libtfc_sp.so

libtfc_wp.so

libtfc_fmk.so

libtfc_acloc.so

libtfc_async.so

libtfc_event_filter.so

libtls77-m.so

libmcc_exec.so

libmcc_sys.so

libDStringTeMIP.so

libstd_v2.so.1

libCsup.so.1

libpthread.so.1

libm.so.1

libunwind.so.1

libc.so.1

libdl.so.1

shared library binding:

deferred

global hash table disabled

global hash table size 1103

shared library mapped private disabled

shared library segment merging disabled

shared vtable support disabled

explicit unloading disabled

linkage table protection disabled

segments:

index type address flags size

8 text 4000000000000000 z---c- D (default)

9 data 6000000000000000 ---m-- D (default)

executable from stack: D (default)

kernel assisted branch prediction enabled

lazy swap allocation for dynamic segments disabled

nulptr references enabled

address space model: default

caliper dynamic instrumentation disabled

new values:

64-bit ELF executable

shared library dynamic path search:

LD_LIBRARY_PATH enabled first

SHLIB_PATH enabled second

embedded path enabled third /opt/vzb/dstring/lib:/usr/opt/temip/lib:/usr/opt/temip/CMA/lib

shared library list:

libtfc_sp.so

libtfc_wp.so

libtfc_fmk.so

libtfc_acloc.so

libtfc_async.so

libtfc_event_filter.so

libtls77-m.so

libmcc_exec.so

libmcc_sys.so

libDStringTeMIP.so

libstd_v2.so.1

libCsup.so.1

libpthread.so.1

libm.so.1

libunwind.so.1

libc.so.1

libdl.so.1

shared library binding:

deferred

global hash table disabled

global hash table size 1103

shared library mapped private enabled

shared library segment merging disabled

shared vtable support disabled

explicit unloading disabled

linkage table protection disabled

segments:

index type address flags size

8 text 4000000000000000 z---c- D (default)

9 data 6000000000000000 ---m-- D (default)

executable from stack: D (default)

kernel assisted branch prediction enabled

lazy swap allocation for dynamic segments disabled

nulptr references enabled

address space model: default

caliper dynamic instrumentation disabled





bash-3.00$ gdb /opt/vzb/bin/vt_tl1_am -pid 20361

HP gdb 5.8 for HP Itanium (32 or 64 bit) and target HP-UX 11.2x.

Copyright 1986 - 2001 Free Software Foundation, Inc.

Hewlett-Packard Wildebeest 5.8 (based on GDB) is covered by the

GNU General Public License. Type "show copying" to see the conditions to

change it and/or distribute copies. Type "show warranty" for warranty/support.

..

Attaching to program: /opt/vzb/bin/vt_tl1_am, process 20361



warning: The shared libraries were not privately mapped; setting a

breakpoint in a shared library will not work until you rerun the program;

stepping over longjmp calls will not work as expected.

Use the following command to enable debugging of shared libraries.

chatr +dbg enable



warning: Load module /usr/opt/temip/lib/libclntsh.so.9.0 has been stripped.

Debugging information is not available.





warning: Load module /usr/opt/temip/lib/libwtc9.so has been stripped.

Debugging information is not available.



0xc00000000028ad30:0 in _select_sys+0x30 () from /usr/lib/hpux64/libc.so.1

(gdb)
(gdb) bt

#0 0xc00000000028ad30:0 in _select_sys+0x30 () from /usr/lib/hpux64/libc.so.1

#1 0xc00000000029d380:0 in select+0xc0 () from /usr/lib/hpux64/libc.so.1

#2 0xc000000002a5e7e0:0 in mcc_sys_select_a () at mcc_sys_file.c:282

#3 0xc000000007ccffa0:0 in im__rpc_server () at ../mcc_ipc_unix.c:2603

#4 0xc000000007ccfa00:0 in mcc_start_server () at ../mcc_main_sub.c:344

#5 0x400000000054c6c0:0 in main () at mcc_mmshell_unix.c:483

(gdb) p *this

No symbol "this" in current context.

(gdb) cont

Continuing.



Program received signal 0, Signal 0

si_code: 0 - .

[Switching to thread 26 (system thread 7139865)]

0xc00000000028e190:0 in _nanosleep_sys+0x30 () from /usr/lib/hpux64/libc.so.1

(gdb) break vt_tl1am_comm.cxx:1103

Breakpoint 1 at 0x4000000000b3ae60:0: file ../tl1_am/src/vt_tl1am_comm.cxx, line 1103 from /opt/vzb/bin/vt_tl1_am.

(gdb) c

Continuing.



Program received signal SIGSEGV, Segmentation fault

si_code: 1 - SEGV_MAPERR - Address not mapped to object.

[Switching to thread 41 (system thread 7181009)]

MDequeElement::IsRoot (this=0x9fffffffbf2526f0)

at /usr/opt/temip/tfc/include/tfc_sp_queue.hxx:201

201 MCC_T_Boolean IsRoot() const { return MCC_K_FALSE;}

(gdb) bt

#0 MDequeElement::IsRoot (this=0x9fffffffbf2526f0)

at /usr/opt/temip/tfc/include/tfc_sp_queue.hxx:201

#1 0xc00000000be776a0:0 in MClass::OnMccCall () at ../tfc_sp_queue.hxx:91

#2 0xc00000000be85640:0 in DispatchModuleSM () at ../tfc_fmk_self_mgt.cxx:103

#3 0xc000000007c89320:0 in im__dispatch_local () at ../mcc_dispatch.c:187

#4 0xc000000007cd30c0:0 in im__rpc_thread_main () at ../mcc_ipc_unix.c:4638

#5 0xc000000007bef7b0:0 in thread_shell () at ../mcc_fw_threads.c:488

#6 0xc00000000030f320:0 in __pthread_bound_body+0x190 ()

from /usr/lib/hpux64/libpthread.so.1

(gdb) p *this

Error accessing memory address 0x33fffffff0: Bad address.

(gdb) exit

The program is running. Quit anyway (and detach it)? (y or n) y

Detaching from program: /opt/vzb/bin/vt_tl1_am, process 20361

bash-3.00$ exit

Yes MCC_K_FLASE is a constant which means false.

[Dennis Handly]

>-mt +DD64 +z -AA ...

Are all of the aC++ shlibs in the chatr output compiled with -mt?

>chatr +dbg enable /home/dholt/vt_tl1_am
>shared library mapped private enabled
>$ gdb /opt/vzb/bin/vt_tl1_am -pid 20361
>warning: The shared libraries were not privately mapped;

Did you restart vt_tl1_am after you did the chatr?

>(gdb) p *this
>No symbol "this" in current context.

You need to do this when you get the signal.

>(gdb) p *this
>Error accessing memory address 0x33fffffff0: Bad address.

This isn't good. But why did it say: (this=0x9fffffffbf1c26f0)

>Yes MCC_K_FLASE is a constant which means false.

I need to see the machine code. Where is this output:
You can also disassemble the function:
(gdb) disas
(gdb) info reg

>>Do you have a thread stack overflow?

(gdb) frame 0
(gdb) p /x $save_sp = $sp
(gdb) frame 6
(gdb) p $save_sp - $sp

[FRESNEDA Christine]

Yes all shared libraries are built with -mt option.

No we haven't restarted the executable after running chatr.

We are currently collecting the other information

Thank you
Christine

[Dennis Handly]

>No we haven't restarted the executable after running chatr.

Don't you have to do that since it aborted when you were in gdb?

[Suganya Thirumurthy]

Christine,

Could you please try again with chatr'ed executable as mentioned by Dennis and let us know whether could you reproduce the crash again?

Please mail to wdb-help@cup.hp.com to get quick replies for your queries/issues related to debugger.

Thanks
Suganya T

[FRESNEDA Christine]

Hello,

all requested information sent via email to wdb-help@cup.hp.com.

Thank you again for your help.
I will post the solution in this thread.

regards,
Christine