C program just stops...

[Carlo Henrico_1]

Hi all

We have a C program which prior to implementing SSL on Oracle worked just fine. We have now installed SSL and for long period of time the program works fine but occassionally it just terminates with no errors/warnings, just a coredump. The debugger info on the coredump is as follows:

#0 0xc0213f60 in kill+0x10 () from /usr/lib/libc.2

#1 0xc01ab4f4 in raise+0x24 () from /usr/lib/libc.2

#2 0xc01f0c28 in abort_C+0x160 () from /usr/lib/libc.2

#3 0xc01f0c84 in abort+0x1c () from /usr/lib/libc.2

#4 0xc01ab4f4 in raise+0x24 () from /usr/lib/libc.2

#5 0xc4cf691c in xehInterpretSavedSigaction+0x2d8 () from /opt/mqm/lib/libmqmcs_r.sl

#6 0xc4cf6f18 in xehExceptionHandler+0x4e4 () from /opt/mqm/lib/libmqmcs_r.sl

#7

#8 0xcd9316f0 in nzossc_set_cert+0x80 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#9 0xcd931e58 in nzossp_set_persona+0x1e4 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#10 0xcd932380 in nzosscc_set_Client_Credential+0xd0 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#11 0xcd932530 in nzosSetCredential+0x80 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#12 0xcdada8fc in ntzCreateConnection+0x5e8 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#13 0xcdada138 in ntzconnect+0x3e0 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#14 0xcd973ba8 in ntconn+0xf4 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#15 0xcd912608 in nsopen+0x135c () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#16 0xcd8f7e8c in nscall1+0x340 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#17 0xcd8f7114 in nscall+0x22c () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#18 0xcd9213f0 in niotns+0xe94 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#19 0xcd976518 in nigcall+0x48 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#20 0xcd924328 in osncon+0x274 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#21 0xcd6a756c in kpuadef+0x44 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#22 0xcd724b9c in upiini+0x3a4 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#23 0xcd70c13c in upiah0+0xa8 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#24 0xcd6a7334 in kpuatch+0x7e8 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#25 0xcd6e98dc in OCIServerAttach+0x94 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

#26 0xc48396b4 in oracle::occi::ConnectionImpl::openConnection+0x17c () from /ora927/app/product/920/lib32/libocci.sl

#27 0xc4838084 in oracle::occi::ConnectionImpl::ConnectionImpl+0xf0 () from /ora927/app/product/920/lib32/libocci.sl

#28 0xc483237c in oracle::occi::EnvironmentImpl::createConnection+0x7c () from /ora927/app/product/920/lib32/libocci.sl

#29 0xc47927a8 in ccOraConnect::connect (this=0x40a06320, pszUserId=0x4044cd62 "crisp1", pszPassword=0x4044cd7a "1", pszDBMS=0x4044cd8e "ora927") at /app/projects/common/envN/oraoccilib/ccOraConnect.cpp:80

#30 0x2f33e4 in CTHREAD_OraConnect (pszUser=0x4044cd62 "crisp1", pszPW=0x4044cd7a "1", pszDBMS=0x4044cd8e "ora927", profilerEnabled=0) at /app/projects/common/envN/include/cthread.cpp:165

#31 0x43f694 in clsRouter::connect (this=0x41967dd0) at /app/projects/rtgs/envN/rtgs_server/clsRouter.cpp:75

#32 0x43fbb0 in clsRouter::EntryPoint (pArg=0x41967dd0) at /app/projects/rtgs/envN/rtgs_server/clsRouter.cpp:180

#33 0xc005375c in __pthread_bound_body+0xa8 () from /usr/lib/libpthread.1

#34 0xc0080784 in __pthread_bound_start+0x14 () from /usr/lib/libpthread.1


Has anybody got any ideas please? I am totally lost with this one.

Thanks

Carlo

[Carlo Henrico_1]

Sorry - I neglected to metion some important info!

HP-UX 11-23
Oracle 9.2.0.7

Carlo

[Steven E. Protter]

Shalom Carlo,

Is the SSL on the system properly patched/revision and configured?

Is Oracle's SSL implementation properly configured?

Perhaps use tusc to get more data on the process and make sure the OS isn't causing issues.

SEP

[Carlo Henrico_1]

Hi Steven

I have used the 9.2.0.7 patchset, or are you referring to something in HP-UX?

The connection and processing normally works for periods of time, it is just occassionally that the program abends/coredumps.

I will look in to tusc...

Carlo

[IT Response]

Hi Carlo,
Based on
#8 0xcd9316f0 in nzossc_set_cert+0x80 () from /ora927/app/product/920/lib32/libclntsh.sl.9.0

Oracle routine nzossc_set_cert trows a signal for some reason. It could be bounds violation, bus error and so forth. We need to have the core inside to find out why, or better off you can pass the question to Oracle that own the problem, they probably have seen this before.

cheers,
Nick

[Dennis Handly]

As Nick says, the function nzossc_set_cert is getting a signal. By decompiling the code we could attempt to guess what it was.

(gdb) frame 8
(gdb) disas nzossc_set_cert nzossc_set_cert+0x80+8
(gdb) info reg

I suppose it is possible you have a thread stack overflow?? You could estimate the thread stack size by:
frame 8
p /x $save_sp = $sp
frame 34
p $save_sp - $sp

The default on PA is 64 Kb.