core dump using pam authentication

[Dennis Handly]

>the address seems to be the same.

That's good.

>There is a way to run the check outside the debugger?

Where would this be? In your application?
Since you know the address you could add statements that did the checking:
if (*(void**)0x6000000000286388L == (void*)0x776f72643a200000L)

If you think the address may change, you could initialize a variable and if not correct, use gdb to change it:
void *gdb_watch = (void*)0x6000000000286388L;

Unfortunately, you would have to make sure that the address 0x6000000000286388 was valid before you dereferenced it. Possibly by doing:
if (sbrk(0) > gdb_watch && *(void**)gdb_watch == (void*)0x776f72643a200000L)

If you are trying to debug heap corruption, have you looked into gdb's:
set heap-check on string on

[Marco Salvi]

Dennis, the addres is not so constant.

I tried this that seemed able to help me

export these variables

BATCH_RTC=on

LD_PRELOAD=/opt/langtools/lib/hpux64/librtc.sl

crete a rtcconfig file

and run my program.

If a kill it in the middle a report about memory is generated, but if I wait until the program crash by itself I get this message

# [ timer expired, abort... ]

[Dennis Handly]

>the address is not so constant.

Then watch points won't help.

>I tried this that seemed able to help me ... BATCH_RTC=on

Any reason you couldn't use gdb and set heap-check directly?
Also what version of wdb do you have? Have you downloaded the latest, 5.7?
http://www.hp.com/go/wdb

>but if I wait until the program crash by itself I get this message
# [ timer expired, abort... ]

If it crashes before it prints an error, then that won't help.

For the "timer" message, you may have contact wdb-help@cup.hp.com.
(See the "HP WDB technical support" link.)

[Dennis Handly]

>So I think that the memory has been corrupted by the function pam_authenticate that pass the structure: struct pam_message **msg
>and the value of: msg[0]->msg is "Password: "

You might want to print the values of msg, msg[0] .. msg[0]+num_msg-1 and then msg[0][i].msg.
Also the value num_msg.
If these are close to your corrupted heap address, that may be a clue.

There seems to be a data structure inconsistency between struct pam_message **msg and struct pam_response **resp.

The second seems to be a pointer to an array of pam_response. The first seems to be a pointer to an array of pointers to pam_message. I.e. for msg, there is no reason for a ** if all you want to do is read and not change them.

Ok, I was confused by your syntax. From pam_start(3) you need to use this to index through each msg:
msg[0][i].msg

BTW: pam_start(3) says that msg should be:
const struct pam_message **msg

[F Verschuren]

last time (4 years ago) I saw this errors It workt to unconvert the system (after that the script was working.)

after converting again the script kept working, never knowed why but reading this lines it makes sens sind's you are telling it is a mem problem, make the system untrusted and trusted again removes the pam from memory....

if you are able to do this please try.

ps be aware that it you have max passwd lenkt on 8 caracters that after unconverting a user that is thinking he is using 9 caracters can not log in anny more :)

[Marco Salvi]

Hi,
I tried to work also disabling the trusted mode but the error is still there.

[Marco Salvi]

Unfortunately not yet able to fix this problem. Just one more question.

Could this problem be due to a stack overflow?

Actually the string "Password :" is stored in a location in the stack, but I have the chunk " word:" in the heap.

I'm not really into the memory model of HP Itanium so I apologize in case it is a very stupid question.

[Dennis Handly]

>Could this problem be due to a stack overflow?

That's a good question because a signal 11 can occur just about anywhere.

But in this case, the address is a load and $r32 is a bogus address, ascii string.

[Marco Salvi]

Hi Dennis,
I've been able to run my program for a while whit check-heap and check-heap string.

(gdb) cont
Continuing.
[Switching to thread 60 (system thread 8729618)]
warning: Memory block (size = 260 address = 0x6000000001fc0630) appears to be corrupted at the end.
Allocation context not found

#1 pam_sm_authenticate() from /usr/lib/security/hpux64/libpam_unix.so.1
#2 pam_authenticate() from /usr/lib/hpux64/libpam.so.1
#3 PamUnixRegistryImpl_checkPassword() from /home/tws83/appserver/lib/ext/libtwspamjni.so
#4 Java_com_ibm_tws_pam_security_registry_PamUnixRegistryImpl_pam_1authenticate() from /home/tws83/appserver/lib/ext/libtwspamjni.so
warning: Use command backtrace (bt) to see the current context.

Ignore top 4 frames belonging to leak detection library of gdb.

__rtc_event () at ../../../Src/gnu/gdb/infrtc.c:1329
1329 in ../../../Src/gnu/gdb/infrtc.c
(gdb) x/16gx 0x6000000001fc0630 -16
0x6000000001fc0620: 0xfeedfacefeedface 0xfeedfacefeedface
0x6000000001fc0630: 0x000000019fffffff 0xd94013709fffffff
0x6000000001fc0640: 0xd94013709fffffff 0xd94013789fffffff
0x6000000001fc0650: 0xd94013789fffffff 0xd94013809fffffff
0x6000000001fc0660: 0xd94013809fffffff 0xd94013889fffffff
0x6000000001fc0670: 0xd94013889fffffff 0xd94013909fffffff
0x6000000001fc0680: 0xd94013909fffffff 0xd95008409fffffff
0x6000000001fc0690: 0xd95008409fffffff 0xd95008509fffffff
(gdb)
0x6000000001fc06a0: 0xd95008509fffffff 0xd95008609fffffff
0x6000000001fc06b0: 0xd95008609fffffff 0xd95008709fffffff
0x6000000001fc06c0: 0xd95008709fffffff 0xd95008809fffffff
0x6000000001fc06d0: 0xd95008809fffffff 0xd95008909fffffff
0x6000000001fc06e0: 0xd95008909fffffff 0xd95008a09fffffff
0x6000000001fc06f0: 0xd95008a09fffffff 0xd95008b09fffffff
0x6000000001fc0700: 0xd95008b09fffffff 0xd9400fb89fffffff
0x6000000001fc0710: 0xd9400fb89fffffff 0xd9400fc09fffffff
(gdb)
0x6000000001fc0720: 0xd9400fc09fffffff 0xd9400fc89fffffff
>>0x6000000001fc0730: 0xd9400fc850617373 0x776f72643a200020
0x6000000001fc0740: 0x6000000001fc1170 0x6000000001fc0a90
0x6000000001fc0750: 0x6000000001fc06d0 0x0000000000000000
0x6000000001fc0760: 0x6000000001fc1ee0 0x0000000000000000
0x6000000001fc0770: 0x6000000001aa8450 0x000000160000004b
0x6000000001fc0780: 0x6000000001fc0720 0x6000000001fc0770
0x6000000001fc0790: 0x6000000001fc0a90 0x6000000001fc0a90

The line with the leading >> seem to be corrupted by the string "Password: ".

There any way I can watch all the read/write on that kind of allocation?

It seems some dynamic structure allocated in pam_sm_authenticate but I don't have access to this code so it is not really simple to debug

[Dennis Handly]

>I've been able to run my program for a while with check-heap and check-heap string.

I would expect it to be blasted by a string operation. Of course that could be inlined and not show up.

>warning: Memory block (size = 260 address = 0x6000000001fc0630) appears to be corrupted at the end.

Yes. But it seems the whole block is bogus since there seems to be a 8 byte pattern that is half there, and missing 9fffffff. (Unless the pointers have been shifted down 4 bytes: 9fffffffd9401370)

>The line with the leading >> seem to be corrupted by the string "Password: ".

Yes.

> There any way I can watch all the read/write on that kind of allocation?

If you use "help set heap-check" there is something about monitoring addresses but if the address isn't the same, you'll have problems. Otherwise if it is the same address you can put a watchpoint on it.

Who printed this poor stack trace? Where are the hex addresses and function offsets?
#1 pam_sm_authenticate() .../libpam_unix.so.1
#2 pam_authenticate() .../libpam.so.1
#3 PamUnixRegistryImpl_checkPassword() .../libtwspamjni.so

Can you get a better one with "bt"?

You might want to disassembly around pam_sm_authenticate so you can figure out was the last call made. (Does #0 exist?)

If it was strcpy/memcpy that may be the one that moved "Password". If free, the corruption was made sooner.