Getting SEGV_ACCERR, problem gone after reboot

[Predrag Maksimovic]

The platform in question is HP-UX 11.23 Itanium 64bit.

When we install our product and run it, it crashes with SIGSEGV, SEGV_ACCERR.

When we exclude the portion of the code where the crash happens and replace it with some dummy code, the SEGV_ACCERR happens somewhere else. The program seems to be unstable until we reboot. When we reboot the machine, the same program starts working fine. Also, the port of this code works fine both without and with reboot on other platforms we use, including HP-UX PA RISC 32bit.

Any idea of what could cause this signal, and particularly what could cause it to go away after we reboot?

Our program is a java process running under su account. It accepts RMI requests and calls into the native code via JNI. The signal happens in the native code, and consequently the JVM crashes.

Here is the signal info from the JVM crash log:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (11) at pc=c0000000000f2671, pid=17640, tid=258
#
# Java VM: Java HotSpot(TM) 64-Bit Server VM (1.4.2 1.4.2.10-060112-22:17-IA64W IA64W mixed mode)
# Problematic frame:
# C [libpthread.so.1+0x4f671] +0xffffffff
#

--------------- T H R E A D ---------------

Current thread (60000000026c9050): JavaThread "Thread-19" daemon [_thread_in_native, id=258, lwp_id=108688]

siginfo:
si_signo [11]: SIGSEGV 11* segmentation violation
si_errno [0]: Error 0
si_code [2]: SEGV_ACCERR [addr: 0x30]
si_addr=0000000000000030

Here is a portion of the stack which shows the error in one scenario. Note, when I modify the code, the SEGV_ACCERR happens elsewhere.

Program terminated with signal 6, Aborted.

Bad function get_java_mutex_info

#0 0xc00000000032f890:0 in kill+0x30 () from /usr/lib/hpux64/libc.so.1
#1 0xc0000000002541d0:0 in raise+0x30 () from /usr/lib/hpux64/libc.so.1
#2 0xc0000000002f0f90:0 in abort+0x190 () from /usr/lib/hpux64/libc.so.1
#3 0xc0000000194a2a40:0 in os::abort ()
at /CLO/Components/JAVA_HOTSPOT/Src/src/os/hp-ux/vm/os_hp-ux.cpp:3227
#4 0xc0000000196f7880:0 in VMError::report_and_die ()
at /CLO/Components/JAVA_HOTSPOT/Src/src/share/vm/utilities/vmError.cpp:781
#5 0xc0000000194abb70:0 in os::Hpux::JVM_handle_hpux_signal ()
at /CLO/Components/JAVA_HOTSPOT/Src/src/os_cpu/hp-ux_ia64/vm/os_hp-ux_ia64.cpp:729
#6 0xc0000000194a14a0:0 in os::Hpux::signalHandler ()
at /CLO/Components/JAVA_HOTSPOT/Src/src/os/hp-ux/vm/os_hp-ux.cpp:2836
#7
#8 0xc0000000000f2670:1 in + 0x111 ()
from /usr/lib/hpux64/libpthread.so.1
#9 0xc0000000000f1fd0:0 in __tls_get_addr+0x2e0 ()
from /usr/lib/hpux64/libpthread.so.1
warning: No unwind information found.
Skipping this library /usr/lib/hpux64/libcl.so.1.

#10 0xc000000001266790:0 in __cxa_get_globals+0x50 ()
from /usr/lib/hpux64/libCsup.so.1
#11 0xc00000000126d000:0 in std::uncaught_exception()+0x20 ()
from /usr/lib/hpux64/libCsup.so.1
#12 0xc00000000126cce0:0 in __cxa_vec_dtor+0x60 ()
from /usr/lib/hpux64/libCsup.so.1

[Dennis Handly]

>when I modify the code, the SEGV_ACCERR happens elsewhere.

(If this also has something like frame #8 to #12, it is the same thing.)

I was going to say contact the Response Center to look into the Java abort but it appears the problem is a user one, if you were out of space?

#8&9: It appears libpthread is having problems and is aborting after calling __tls_get_addr. You might want to get a newer libpthread patch.

#11: The aC++ runtime is trying to see if it is being called during cleanup.

#12: You were starting to destroy an array and #8 failed. If there was a throw while constructing the array (#13), this could also cause it.

#13: Do you have more frames??

[Predrag Maksimovic]

Thank you for the reply.

Yes there are more frames before #12, but those change. As a matter of fact, I was getting SEGV_ACCERR even without libpthread and __cxa_vec_dtor() in the stackframe. The only thing that always stays the same is the portion #0-#7, but that's the signal handling part, meaning after the fact.

Any idea what would cause the SEGV_ACCERR to go away (or seemingly go away) after we reboot and run the exact same program? If the problem was with libpthread, I would have expected the crashes to continue after the reboot.

Also, can somebody please explain what exactly is the difference between SEGV_ACCERR and SEGV_MAPERR?

[Dennis Handly]

>Yes there are more frames before #12, but those change.

It is important to see them. Also important to get an idea of the thread stack size.

>I was getting SEGV_ACCERR even without libpthread and __cxa_vec_dtor() in the stackframe.

This could be a thread stack overflow. It would help to see the disassembly on #8:
(gdb) frame 8
(gdb) disas 0xc0000000000f2670-16*8 0xc0000000000f2670+16*4
(gdb) info reg

>The only thing that always stays the same is the portion #0-#7, but that's the signal handling part, meaning after the fact.

Right, pretty useless.

>can somebody please explain what exactly is the difference between SEGV_ACCERR and SEGV_MAPERR?

I suppose the first is the fact that the access rights are incorrect. Probably writing to the guard page. The latter could be an address that isn't valid?

In any case it doesn't matter. You need to look at the instructions and register values to determine the problem.