core dump using pam authentication

[Dennis Handly]

>At the moment I've been able to recreate it in more simple program.

That's good.

> #1 pam_sm_authenticate .../hpux64/libpam_unix.so.1

Can you use "bt" to get a real stack trace? It would be nice to know the offset in pam_sm_authenticate.

[Marco Salvi]

Running bt after this memory corruption information return something like this

warning: Memory block (size = 260 address = 0x60000000012e7af0) appears to be corrupted at the end.
Allocation context not found

#1 pam_sm_authenticate() from /usr/lib/security/hpux64/libpam_unix.so.1
#2 pam_authenticate() from /usr/lib/hpux64/libpam.so.1
#3 PamUnixRegistryImpl_checkPassword() from /local/tws83/appserver/lib/ext/libtwspamjni.so
#4 Java_com_ibm_tws_pam_security_registry_PamUnixRegistryImpl_pam_1authenticate() from /local/tws83/appserver/lib/ext/libtwspamjni.so
warning: Use command backtrace (bt) to see the current context.

Ignore top 4 frames belonging to leak detection library of gdb.

__rtc_event () at ../../../Src/gnu/gdb/infrtc.c:1329
1329 ../../../Src/gnu/gdb/infrtc.c: No such file or directory.
in ../../../Src/gnu/gdb/infrtc.c
Current language: auto; currently c
(gdb) bt
#0 __rtc_event () at ../../../Src/gnu/gdb/infrtc.c:1329
#1 0x87ffffffbf73ee60:0 in check_bounds (pointer=0x60000000012e7af0,
size=260, pclist=0x600000000104c568) at ../../../Src/gnu/gdb/infrtc.c:1437
#2 0x87ffffffbf743ea0:0 in rtc_record_free ()
at ../../../Src/gnu/gdb/infrtc.c:2463
#3 0x87ffffffbf7458a0:0 in free () at ../../../Src/gnu/gdb/infrtc.c:2789
#4 0x87ffffff7f3b8c60:0 in + 0x40 ()
from /usr/lib/security/hpux64/libpam_unix.so.1
#5 0x87ffffffbde07670:0 in pam_end+0x280 () from /usr/lib/hpux64/libpam.so.1

I'm wondering in there is a way to lock on this 260 bytes of memory in order to discover what is corrupting it

[Dennis Handly]

>Allocation context not found
#1 pam_sm_authenticate() .../libpam_unix.so.1

This must be where it is allocated. My gdb 5.7.4 says: Allocation context might not be correct.

The only calloc in pam_sm_authenticate allocates a 260 byte struct:
typedef struct _unix_auth_data_ {
int key_status;
char netname[255+1];
} unix_auth_data;

The pointer is stored in some magic structure set/get by pam_get_data/pam_set_data, part of pam_handle_t.

#4 0x87ffffff7f3b8c60:0 in + 0x40

This is probably unix_cleanup, who cleans up the pam_set_data linked list, containing those unix_auth_data.

>I'm wondering in there is a way to lock on this 260 bytes of memory in order to discover what is corrupting it

Well you can set a hardware watch point if the address doesn't change:
(gdb) watch *(char (*) [260])0x60000000012e7af0

Have you tried pam debugging?
Do a google search for: /etc/pam_debug site:hp.com

For some reason the pdf link isn't there but the "View as HTML" link gives it: ...
To enable tracing the native PAM library, turn on debugging on syslog level
(the steps needed may vary for the different UNIX flavors). The related service
name is auth. To turn on tracing, configure syslogd as in the following
Page 156
Configuration Value Pack 3.0.3 Installation and Administration Guide

example:
# touch /etc/pam_debug
# vi /etc/syslog.conf
auth.debug /tmp/pam_auth.log

[Dennis Handly]

It looks like they may have figured it out. There was a strcpy to this field:
char netname[255+1];

But the source was uninitialized.

[Marco Salvi]

Thank you Dennis for all your help and

Merry Christmas