topic Re: core dump using pam authentication in Operating System - Linux

core dump using pam authentication

Marco Salvi — Tue, 27 Nov 2007 06:42:57 GMT

Hi,
I'm experiencing a core dump in a java application using JNI to call the pam module to authenticate a user.
Scanning the core I've noticed that the memory corruption is caused by the following string.

"word: "

I was able to find this word in the messages received by my conversation function. This word should be a substring of

"Password: "

I've tried to apply the latest patches available for pam but that does not fix the problem.

I read that pam call are not thread safe.
In my application there is a synchronize statement in the java code that contain the code for all the pam call, so it is not possible to have two pam calls executed at the same time in different threads.

Some chunck of code for giving you a better idea

java code
...

synchronized( PamUnixRegistryImpl.class )
{
if (pam_authenticate(s, s1) == 0) {
PasswordCheckFailedException passwordcheckfailedexception =
new PasswordCheckFailedException("Pam Authentication failed for user: " + s);
throw passwordcheckfailedexception;
}
}

...

JNI code

pam_authenticate simply call

PamUnixRegistryImpl_checkPassword (...

C code

....

extern int convert(int num_msg,struct pam_message **msg,struct
pam_response **resp,void *appdata_ptr)
{
// Initialize PAM response object and set password
struct pam_response *temp;
temp = (struct pam_response *)calloc(num_msg,sizeof(struct pam_response));
temp[0].resp_retcode = 0;
temp[0].resp = strdup((const char*)appdata_ptr);
*resp = temp;
return PAM_SUCCESS;
}
static struct pam_conv conv = {convert,NULL};

int PamUnixRegistryImpl_checkPassword (const char *username,const char *password)
{
pwd_buf_t buf;
const char *epass;
struct passwd *pwd;
long last_update, max_age, min_age;
time_t now;
int local_user;
pam_handle_t *pamh=NULL;
int retval;
struct pam_response *pp=NULL;
conv.appdata_ptr = password;
local_user = 0;

// initialize PAM
printf("pam_checkPassword\n");
fflush(stdout);

retval = pam_start("checkpassword", username, &conv, &pamh);

if (retval == PAM_SUCCESS)
{
retval = pam_authenticate(pamh, PAM_SILENT);
printf("pam_authenticate\n");
fflush(stdout);
if (retval != PAM_SUCCESS)
{
pam_end(pamh,retval);
pamh = NULL;
return 0;
}
// End PAM Session
if (pam_end(pamh,retval) != PAM_SUCCESS)
{
pamh = NULL;
return 0;
}
}
else
{
try a local authentication without PAM
}
return 1;
...

Re: core dump using pam authentication

Steven E. Protter — Tue, 27 Nov 2007 07:22:13 GMT

Shalom,

If you want your code debugged, move to the next response because I don't understand your code.

I'd take some time to investigate your /etc/pam.d/ authentication. Make sure PAM is up to date and the system has a recent bi-annual patch.

I'd also make sure Java itself is updated to a recent, stable release.

Any of these factors can cause problems. I'd also be interested to know which pam authentication module your code intends to use to authenticate. Your system may need changes to /etc/nsswitch.conf to accommodate this.

SEP

Re: core dump using pam authentication

Marco Salvi — Tue, 27 Nov 2007 08:06:03 GMT

My system has trusted mode enabled.

I think the pam configuration is ok because the system work for a while before crashing. Java patch and PAM patches are OK (at least I think).

I forgot to say that I'm running my code on hp 11.23 v2 on IA64. The code is 64 bit.

Re: core dump using pam authentication

Marco Salvi — Tue, 27 Nov 2007 08:06:07 GMT

My system has trusted mode enabled.
Pam is actually use to authenticate OS user only.

I think the pam configuration is ok because the system work for a while before crashing. Java patch and PAM patches are OK (at least I think).

I forgot to say that I'm running my code on hp 11.23 v2 on IA64. The code is 64 bit.

Re: core dump using pam authentication

Dennis Handly — Tue, 27 Nov 2007 11:10:48 GMT

>I'm experiencing a core dump in a java application using JNI to call the pam module to authenticate a user.

Do you have a stack trace that points to the line in your source above?

>Scanning the core I've noticed that the memory corruption is caused by the following string. "word: "
>I was able to find this word in the messages received by my conversation function.

What do you mean by scanning? You must debug the problem and print out variables.

Re: core dump using pam authentication

Marco Salvi — Tue, 27 Nov 2007 11:36:09 GMT

#0 0xc00000000490b670:0 in real_malloc+0x670 () from /usr/lib/hpux64/libc.so.1
#1 0xc00000000490ae10:0 in _malloc+0x7d0 () from /usr/lib/hpux64/libc.so.1
#2 0xc000000004915fb0:0 in malloc+0x140 () from /usr/lib/hpux64/libc.so.1
#3 0x9fffffffbf0b1770:0 in nnfun2a+0x310 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#4 0x9fffffffbf0b0ec0:0 in nnfsn2a+0x80 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#5 0x9fffffffbf0820b0:0 in niqname+0x350 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#6 0x9fffffffbeeb9c40:0 in kwfnran+0x220 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#7 0x9fffffffbee549f0:0 in kwfcinit+0x210 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#8 0x9fffffffc0372440 in + 0xef0 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#9 0x9fffffffbec5ec40:0 in OCIServerAttach+0xe0 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1

this is the top of the stack trace

disassembling the part that fails I have noticed

0xc00000000490b660:0 :
(p6) ld8 r32=[r18]
0xc00000000490b660:1 :
adds r19=-1,r23
0xc00000000490b660:2 :
add r8=r10,r14
0xc00000000490b670:0 :
(p7) ld8.a r16=[r32]

(gdb) x $r32
0x776f72643a200000: Error accessing memory address 0x776f72643a200000: Bad address.

(gdb) x/x $r18
0x6000000000286398: 0x600000000284b338
(gdb) x/16gx $r18 -32
0x6000000000286378: 0x0000000100000000 0x0000000000000120
0x6000000000286388: 0x776f72643a200000 0x0000000000000000
0x6000000000286398: 0x600000000284b338 0x0000000000000062
0x60000000002863a8: 0x0000000000000064 0x0000000100000000
0x60000000002863b8: 0x0000000000000130 0x6000000002f00018
0x60000000002863c8: 0x0000000000000002 0x6000000002f00278
0x60000000002863d8: 0x0000000000000062 0x0000000000000064
0x60000000002863e8: 0x0000000100000000 0x0000000000000140
(gdb) x/16gs $r18 -32
0x6000000000286378: ""
0x6000000000286379: ""
0x600000000028637a: ""
0x600000000028637b: "\001"
0x600000000028637d: ""
0x600000000028637e: ""
0x600000000028637f: ""
0x6000000000286380: ""
0x6000000000286381: ""
0x6000000000286382: ""
0x6000000000286383: ""
0x6000000000286384: ""
0x6000000000286385: ""
0x6000000000286386: "\001 word: "
0x600000000028638f: ""
0x6000000000286390: ""

It seems that the memory has been corrupted by a string

I believe that this is part of "Password: "

So I think that the memory has been corrupted by the function pam_authenticate that pass the structure

struct pam_message **msg

to my conversation function

and the value of

msg[0]->msg is "Password: "

Do you agree with my analysis?

Re: core dump using pam authentication

Dennis Handly — Tue, 27 Nov 2007 12:02:08 GMT

>#0 0xc00000000490b670:0 in real_malloc+0x670 libc.so.1

Wow! Someone that knows how to debug!

>It seems that the memory has been corrupted by a string

Exactly.

>So I think that the memory has been corrupted by the function pam_authenticate that pass the structure struct pam_message **msg to my convert function

Possibly. But your calloc for resp looks ok, provided num_msg isn't 0.

>and the value of msg[0]->msg is "Password: "

I don't see you using msg? Were you suppose to set it?

>Do you agree with my analysis?

Partly. You should put a hardware watchpoint on 0x6000000000286388 to see who is modifying it:
(gdb) watch *(void**)0x6000000000286388

Re: core dump using pam authentication

Marco Salvi — Tue, 27 Nov 2007 12:58:59 GMT

Thanks for your suggestion.
I have tried this but unfortunately the corruption happens after some time. I spend an half an hour pressing enter then the debugger get in stuck.
There is a way to set the watchpoint and tell the debugger to ignore malloc and free for that address?

Re: core dump using pam authentication

Dennis Handly — Tue, 27 Nov 2007 17:18:27 GMT

>I have tried this but unfortunately the corruption happens after some time. I spend an half an hour pressing enter then the debugger get in stuck.

Then you may have problems. Especially if java allocates a different address. Is the corruption address the same in two runs?

>There is a way to set the watchpoint and tell the debugger to ignore malloc and free for that address?

You could add a condition to the watch point to only stop if the value 0x776f72643a200000 is stored:
cond watch-number (*(void**)0x6000000000286388 == 0x776f72643a200000)

Re: core dump using pam authentication

Marco Salvi — Thu, 29 Nov 2007 09:11:22 GMT

Hi Dennis,
the address seems to be the same.
There is a way to run the check outside the debugger?

Re: core dump using pam authentication

Dennis Handly — Thu, 29 Nov 2007 19:34:40 GMT

>the address seems to be the same.

That's good.

>There is a way to run the check outside the debugger?

Where would this be? In your application?
Since you know the address you could add statements that did the checking:
if (*(void**)0x6000000000286388L == (void*)0x776f72643a200000L)

If you think the address may change, you could initialize a variable and if not correct, use gdb to change it:
void *gdb_watch = (void*)0x6000000000286388L;

Unfortunately, you would have to make sure that the address 0x6000000000286388 was valid before you dereferenced it. Possibly by doing:
if (sbrk(0) > gdb_watch && *(void**)gdb_watch == (void*)0x776f72643a200000L)

If you are trying to debug heap corruption, have you looked into gdb's:
set heap-check on string on

Re: core dump using pam authentication

Marco Salvi — Fri, 30 Nov 2007 06:46:20 GMT

Dennis, the addres is not so constant.

I tried this that seemed able to help me

export these variables

BATCH_RTC=on

LD_PRELOAD=/opt/langtools/lib/hpux64/librtc.sl

crete a rtcconfig file

and run my program.

If a kill it in the middle a report about memory is generated, but if I wait until the program crash by itself I get this message

# [ timer expired, abort... ]

Re: core dump using pam authentication

Dennis Handly — Fri, 30 Nov 2007 15:18:48 GMT

>the address is not so constant.

Then watch points won't help.

>I tried this that seemed able to help me ... BATCH_RTC=on

Any reason you couldn't use gdb and set heap-check directly?
Also what version of wdb do you have? Have you downloaded the latest, 5.7?
http://www.hp.com/go/wdb

>but if I wait until the program crash by itself I get this message
# [ timer expired, abort... ]

If it crashes before it prints an error, then that won't help.

For the "timer" message, you may have contact wdb-help@cup.hp.com.
(See the "HP WDB technical support" link.)

Re: core dump using pam authentication

Dennis Handly — Fri, 30 Nov 2007 16:05:22 GMT

>So I think that the memory has been corrupted by the function pam_authenticate that pass the structure: struct pam_message **msg
>and the value of: msg[0]->msg is "Password: "

You might want to print the values of msg, msg[0] .. msg[0]+num_msg-1 and then msg[0][i].msg.
Also the value num_msg.
If these are close to your corrupted heap address, that may be a clue.

There seems to be a data structure inconsistency between struct pam_message **msg and struct pam_response **resp.

The second seems to be a pointer to an array of pam_response. The first seems to be a pointer to an array of pointers to pam_message. I.e. for msg, there is no reason for a ** if all you want to do is read and not change them.

Ok, I was confused by your syntax. From pam_start(3) you need to use this to index through each msg:
msg[0][i].msg

BTW: pam_start(3) says that msg should be:
const struct pam_message **msg

Re: core dump using pam authentication

F Verschuren — Fri, 30 Nov 2007 16:45:28 GMT

last time (4 years ago) I saw this errors It workt to unconvert the system (after that the script was working.)

after converting again the script kept working, never knowed why but reading this lines it makes sens sind's you are telling it is a mem problem, make the system untrusted and trusted again removes the pam from memory....

if you are able to do this please try.

ps be aware that it you have max passwd lenkt on 8 caracters that after unconverting a user that is thinking he is using 9 caracters can not log in anny more :)

Re: core dump using pam authentication

Marco Salvi — Sat, 01 Dec 2007 07:35:45 GMT

Hi,
I tried to work also disabling the trusted mode but the error is still there.

Re: core dump using pam authentication

Marco Salvi — Thu, 06 Dec 2007 15:51:44 GMT

Unfortunately not yet able to fix this problem. Just one more question.

Could this problem be due to a stack overflow?

Actually the string "Password :" is stored in a location in the stack, but I have the chunk " word:" in the heap.

I'm not really into the memory model of HP Itanium so I apologize in case it is a very stupid question.

Re: core dump using pam authentication

Dennis Handly — Fri, 07 Dec 2007 05:47:17 GMT

>Could this problem be due to a stack overflow?

That's a good question because a signal 11 can occur just about anywhere.

But in this case, the address is a load and $r32 is a bogus address, ascii string.

Re: core dump using pam authentication

Marco Salvi — Fri, 07 Dec 2007 13:46:51 GMT

Hi Dennis,
I've been able to run my program for a while whit check-heap and check-heap string.

(gdb) cont
Continuing.
[Switching to thread 60 (system thread 8729618)]
warning: Memory block (size = 260 address = 0x6000000001fc0630) appears to be corrupted at the end.
Allocation context not found

#1 pam_sm_authenticate() from /usr/lib/security/hpux64/libpam_unix.so.1
#2 pam_authenticate() from /usr/lib/hpux64/libpam.so.1
#3 PamUnixRegistryImpl_checkPassword() from /home/tws83/appserver/lib/ext/libtwspamjni.so
#4 Java_com_ibm_tws_pam_security_registry_PamUnixRegistryImpl_pam_1authenticate() from /home/tws83/appserver/lib/ext/libtwspamjni.so
warning: Use command backtrace (bt) to see the current context.

Ignore top 4 frames belonging to leak detection library of gdb.

__rtc_event () at ../../../Src/gnu/gdb/infrtc.c:1329
1329 in ../../../Src/gnu/gdb/infrtc.c
(gdb) x/16gx 0x6000000001fc0630 -16
0x6000000001fc0620: 0xfeedfacefeedface 0xfeedfacefeedface
0x6000000001fc0630: 0x000000019fffffff 0xd94013709fffffff
0x6000000001fc0640: 0xd94013709fffffff 0xd94013789fffffff
0x6000000001fc0650: 0xd94013789fffffff 0xd94013809fffffff
0x6000000001fc0660: 0xd94013809fffffff 0xd94013889fffffff
0x6000000001fc0670: 0xd94013889fffffff 0xd94013909fffffff
0x6000000001fc0680: 0xd94013909fffffff 0xd95008409fffffff
0x6000000001fc0690: 0xd95008409fffffff 0xd95008509fffffff
(gdb)
0x6000000001fc06a0: 0xd95008509fffffff 0xd95008609fffffff
0x6000000001fc06b0: 0xd95008609fffffff 0xd95008709fffffff
0x6000000001fc06c0: 0xd95008709fffffff 0xd95008809fffffff
0x6000000001fc06d0: 0xd95008809fffffff 0xd95008909fffffff
0x6000000001fc06e0: 0xd95008909fffffff 0xd95008a09fffffff
0x6000000001fc06f0: 0xd95008a09fffffff 0xd95008b09fffffff
0x6000000001fc0700: 0xd95008b09fffffff 0xd9400fb89fffffff
0x6000000001fc0710: 0xd9400fb89fffffff 0xd9400fc09fffffff
(gdb)
0x6000000001fc0720: 0xd9400fc09fffffff 0xd9400fc89fffffff
>>0x6000000001fc0730: 0xd9400fc850617373 0x776f72643a200020
0x6000000001fc0740: 0x6000000001fc1170 0x6000000001fc0a90
0x6000000001fc0750: 0x6000000001fc06d0 0x0000000000000000
0x6000000001fc0760: 0x6000000001fc1ee0 0x0000000000000000
0x6000000001fc0770: 0x6000000001aa8450 0x000000160000004b
0x6000000001fc0780: 0x6000000001fc0720 0x6000000001fc0770
0x6000000001fc0790: 0x6000000001fc0a90 0x6000000001fc0a90

The line with the leading >> seem to be corrupted by the string "Password: ".

There any way I can watch all the read/write on that kind of allocation?

It seems some dynamic structure allocated in pam_sm_authenticate but I don't have access to this code so it is not really simple to debug

Re: core dump using pam authentication

Dennis Handly — Sun, 09 Dec 2007 00:31:55 GMT

>I've been able to run my program for a while with check-heap and check-heap string.

I would expect it to be blasted by a string operation. Of course that could be inlined and not show up.

>warning: Memory block (size = 260 address = 0x6000000001fc0630) appears to be corrupted at the end.

Yes. But it seems the whole block is bogus since there seems to be a 8 byte pattern that is half there, and missing 9fffffff. (Unless the pointers have been shifted down 4 bytes: 9fffffffd9401370)

>The line with the leading >> seem to be corrupted by the string "Password: ".

Yes.

> There any way I can watch all the read/write on that kind of allocation?

If you use "help set heap-check" there is something about monitoring addresses but if the address isn't the same, you'll have problems. Otherwise if it is the same address you can put a watchpoint on it.

Who printed this poor stack trace? Where are the hex addresses and function offsets?
#1 pam_sm_authenticate() .../libpam_unix.so.1
#2 pam_authenticate() .../libpam.so.1
#3 PamUnixRegistryImpl_checkPassword() .../libtwspamjni.so

Can you get a better one with "bt"?

You might want to disassembly around pam_sm_authenticate so you can figure out was the last call made. (Does #0 exist?)

If it was strcpy/memcpy that may be the one that moved "Password". If free, the corruption was made sooner.