core dump using pam authentication

[Marco Salvi]

Hi,
I'm experiencing a core dump in a java application using JNI to call the pam module to authenticate a user.
Scanning the core I've noticed that the memory corruption is caused by the following string.

"word: "

I was able to find this word in the messages received by my conversation function. This word should be a substring of

"Password: "

I've tried to apply the latest patches available for pam but that does not fix the problem.

I read that pam call are not thread safe.
In my application there is a synchronize statement in the java code that contain the code for all the pam call, so it is not possible to have two pam calls executed at the same time in different threads.

Some chunck of code for giving you a better idea

java code
...

synchronized( PamUnixRegistryImpl.class )
{
if (pam_authenticate(s, s1) == 0) {
PasswordCheckFailedException passwordcheckfailedexception =
new PasswordCheckFailedException("Pam Authentication failed for user: " + s);
throw passwordcheckfailedexception;
}
}

...

JNI code

pam_authenticate simply call

PamUnixRegistryImpl_checkPassword (...

C code

....

extern int convert(int num_msg,struct pam_message **msg,struct
pam_response **resp,void *appdata_ptr)
{
// Initialize PAM response object and set password
struct pam_response *temp;
temp = (struct pam_response *)calloc(num_msg,sizeof(struct pam_response));
temp[0].resp_retcode = 0;
temp[0].resp = strdup((const char*)appdata_ptr);
*resp = temp;
return PAM_SUCCESS;
}
static struct pam_conv conv = {convert,NULL};

int PamUnixRegistryImpl_checkPassword (const char *username,const char *password)
{
pwd_buf_t buf;
const char *epass;
struct passwd *pwd;
long last_update, max_age, min_age;
time_t now;
int local_user;
pam_handle_t *pamh=NULL;
int retval;
struct pam_response *pp=NULL;
conv.appdata_ptr = password;
local_user = 0;


// initialize PAM
printf("pam_checkPassword\n");
fflush(stdout);

retval = pam_start("checkpassword", username, &conv, &pamh);


if (retval == PAM_SUCCESS)
{
retval = pam_authenticate(pamh, PAM_SILENT);
printf("pam_authenticate\n");
fflush(stdout);
if (retval != PAM_SUCCESS)
{
pam_end(pamh,retval);
pamh = NULL;
return 0;
}
// End PAM Session
if (pam_end(pamh,retval) != PAM_SUCCESS)
{
pamh = NULL;
return 0;
}
}
else
{
try a local authentication without PAM
}
return 1;
...

[Steven E. Protter]

Shalom,


If you want your code debugged, move to the next response because I don't understand your code.

I'd take some time to investigate your /etc/pam.d/ authentication. Make sure PAM is up to date and the system has a recent bi-annual patch.

I'd also make sure Java itself is updated to a recent, stable release.

Any of these factors can cause problems. I'd also be interested to know which pam authentication module your code intends to use to authenticate. Your system may need changes to /etc/nsswitch.conf to accommodate this.

SEP

[Marco Salvi]

My system has trusted mode enabled.

I think the pam configuration is ok because the system work for a while before crashing. Java patch and PAM patches are OK (at least I think).

I forgot to say that I'm running my code on hp 11.23 v2 on IA64. The code is 64 bit.

[Marco Salvi]

My system has trusted mode enabled.
Pam is actually use to authenticate OS user only.

I think the pam configuration is ok because the system work for a while before crashing. Java patch and PAM patches are OK (at least I think).

I forgot to say that I'm running my code on hp 11.23 v2 on IA64. The code is 64 bit.

[Dennis Handly]

>I'm experiencing a core dump in a java application using JNI to call the pam module to authenticate a user.

Do you have a stack trace that points to the line in your source above?

>Scanning the core I've noticed that the memory corruption is caused by the following string. "word: "
>I was able to find this word in the messages received by my conversation function.

What do you mean by scanning? You must debug the problem and print out variables.

[Marco Salvi]

#0 0xc00000000490b670:0 in real_malloc+0x670 () from /usr/lib/hpux64/libc.so.1
#1 0xc00000000490ae10:0 in _malloc+0x7d0 () from /usr/lib/hpux64/libc.so.1
#2 0xc000000004915fb0:0 in malloc+0x140 () from /usr/lib/hpux64/libc.so.1
#3 0x9fffffffbf0b1770:0 in nnfun2a+0x310 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#4 0x9fffffffbf0b0ec0:0 in nnfsn2a+0x80 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#5 0x9fffffffbf0820b0:0 in niqname+0x350 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#6 0x9fffffffbeeb9c40:0 in kwfnran+0x220 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#7 0x9fffffffbee549f0:0 in kwfcinit+0x210 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#8 0x9fffffffc0372440 in + 0xef0 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1
#9 0x9fffffffbec5ec40:0 in OCIServerAttach+0xe0 ()
from /home/oracle/ora10g/lib/libclntsh.so.10.1

this is the top of the stack trace

disassembling the part that fails I have noticed

0xc00000000490b660:0 :
(p6) ld8 r32=[r18]
0xc00000000490b660:1 :
adds r19=-1,r23
0xc00000000490b660:2 :
add r8=r10,r14
0xc00000000490b670:0 :
(p7) ld8.a r16=[r32]


(gdb) x $r32
0x776f72643a200000: Error accessing memory address 0x776f72643a200000: Bad address.

(gdb) x/x $r18
0x6000000000286398: 0x600000000284b338
(gdb) x/16gx $r18 -32
0x6000000000286378: 0x0000000100000000 0x0000000000000120
0x6000000000286388: 0x776f72643a200000 0x0000000000000000
0x6000000000286398: 0x600000000284b338 0x0000000000000062
0x60000000002863a8: 0x0000000000000064 0x0000000100000000
0x60000000002863b8: 0x0000000000000130 0x6000000002f00018
0x60000000002863c8: 0x0000000000000002 0x6000000002f00278
0x60000000002863d8: 0x0000000000000062 0x0000000000000064
0x60000000002863e8: 0x0000000100000000 0x0000000000000140
(gdb) x/16gs $r18 -32
0x6000000000286378: ""
0x6000000000286379: ""
0x600000000028637a: ""
0x600000000028637b: "\001"
0x600000000028637d: ""
0x600000000028637e: ""
0x600000000028637f: ""
0x6000000000286380: ""
0x6000000000286381: ""
0x6000000000286382: ""
0x6000000000286383: ""
0x6000000000286384: ""
0x6000000000286385: ""
0x6000000000286386: "\001 word: "
0x600000000028638f: ""
0x6000000000286390: ""

It seems that the memory has been corrupted by a string

I believe that this is part of "Password: "

So I think that the memory has been corrupted by the function pam_authenticate that pass the structure

struct pam_message **msg

to my conversation function

and the value of

msg[0]->msg is "Password: "

Do you agree with my analysis?

[Dennis Handly]

>#0 0xc00000000490b670:0 in real_malloc+0x670 libc.so.1

Wow! Someone that knows how to debug!

>It seems that the memory has been corrupted by a string

Exactly.

>So I think that the memory has been corrupted by the function pam_authenticate that pass the structure struct pam_message **msg to my convert function

Possibly. But your calloc for resp looks ok, provided num_msg isn't 0.

>and the value of msg[0]->msg is "Password: "

I don't see you using msg? Were you suppose to set it?

>Do you agree with my analysis?

Partly. You should put a hardware watchpoint on 0x6000000000286388 to see who is modifying it:
(gdb) watch *(void**)0x6000000000286388

[Marco Salvi]

Thanks for your suggestion.
I have tried this but unfortunately the corruption happens after some time. I spend an half an hour pressing enter then the debugger get in stuck.
There is a way to set the watchpoint and tell the debugger to ignore malloc and free for that address?

[Dennis Handly]

>I have tried this but unfortunately the corruption happens after some time. I spend an half an hour pressing enter then the debugger get in stuck.

Then you may have problems. Especially if java allocates a different address. Is the corruption address the same in two runs?

>There is a way to set the watchpoint and tell the debugger to ignore malloc and free for that address?

You could add a condition to the watch point to only stop if the value 0x776f72643a200000 is stored:
cond watch-number (*(void**)0x6000000000286388 == 0x776f72643a200000)

[Marco Salvi]

Hi Dennis,
the address seems to be the same.
There is a way to run the check outside the debugger?

[Dennis Handly]

>the address seems to be the same.

That's good.

>There is a way to run the check outside the debugger?

Where would this be? In your application?
Since you know the address you could add statements that did the checking:
if (*(void**)0x6000000000286388L == (void*)0x776f72643a200000L)

If you think the address may change, you could initialize a variable and if not correct, use gdb to change it:
void *gdb_watch = (void*)0x6000000000286388L;

Unfortunately, you would have to make sure that the address 0x6000000000286388 was valid before you dereferenced it. Possibly by doing:
if (sbrk(0) > gdb_watch && *(void**)gdb_watch == (void*)0x776f72643a200000L)

If you are trying to debug heap corruption, have you looked into gdb's:
set heap-check on string on

[Marco Salvi]

Dennis, the addres is not so constant.

I tried this that seemed able to help me

export these variables

BATCH_RTC=on

LD_PRELOAD=/opt/langtools/lib/hpux64/librtc.sl

crete a rtcconfig file

and run my program.

If a kill it in the middle a report about memory is generated, but if I wait until the program crash by itself I get this message

# [ timer expired, abort... ]

[Dennis Handly]

>the address is not so constant.

Then watch points won't help.

>I tried this that seemed able to help me ... BATCH_RTC=on

Any reason you couldn't use gdb and set heap-check directly?
Also what version of wdb do you have? Have you downloaded the latest, 5.7?
http://www.hp.com/go/wdb

>but if I wait until the program crash by itself I get this message
# [ timer expired, abort... ]

If it crashes before it prints an error, then that won't help.

For the "timer" message, you may have contact wdb-help@cup.hp.com.
(See the "HP WDB technical support" link.)

[Dennis Handly]

>So I think that the memory has been corrupted by the function pam_authenticate that pass the structure: struct pam_message **msg
>and the value of: msg[0]->msg is "Password: "

You might want to print the values of msg, msg[0] .. msg[0]+num_msg-1 and then msg[0][i].msg.
Also the value num_msg.
If these are close to your corrupted heap address, that may be a clue.

There seems to be a data structure inconsistency between struct pam_message **msg and struct pam_response **resp.

The second seems to be a pointer to an array of pam_response. The first seems to be a pointer to an array of pointers to pam_message. I.e. for msg, there is no reason for a ** if all you want to do is read and not change them.

Ok, I was confused by your syntax. From pam_start(3) you need to use this to index through each msg:
msg[0][i].msg

BTW: pam_start(3) says that msg should be:
const struct pam_message **msg

[F Verschuren]

last time (4 years ago) I saw this errors It workt to unconvert the system (after that the script was working.)

after converting again the script kept working, never knowed why but reading this lines it makes sens sind's you are telling it is a mem problem, make the system untrusted and trusted again removes the pam from memory....

if you are able to do this please try.

ps be aware that it you have max passwd lenkt on 8 caracters that after unconverting a user that is thinking he is using 9 caracters can not log in anny more :)

[Marco Salvi]

Hi,
I tried to work also disabling the trusted mode but the error is still there.

[Marco Salvi]

Unfortunately not yet able to fix this problem. Just one more question.

Could this problem be due to a stack overflow?

Actually the string "Password :" is stored in a location in the stack, but I have the chunk " word:" in the heap.

I'm not really into the memory model of HP Itanium so I apologize in case it is a very stupid question.

[Dennis Handly]

>Could this problem be due to a stack overflow?

That's a good question because a signal 11 can occur just about anywhere.

But in this case, the address is a load and $r32 is a bogus address, ascii string.

[Marco Salvi]

Hi Dennis,
I've been able to run my program for a while whit check-heap and check-heap string.

(gdb) cont
Continuing.
[Switching to thread 60 (system thread 8729618)]
warning: Memory block (size = 260 address = 0x6000000001fc0630) appears to be corrupted at the end.
Allocation context not found

#1 pam_sm_authenticate() from /usr/lib/security/hpux64/libpam_unix.so.1
#2 pam_authenticate() from /usr/lib/hpux64/libpam.so.1
#3 PamUnixRegistryImpl_checkPassword() from /home/tws83/appserver/lib/ext/libtwspamjni.so
#4 Java_com_ibm_tws_pam_security_registry_PamUnixRegistryImpl_pam_1authenticate() from /home/tws83/appserver/lib/ext/libtwspamjni.so
warning: Use command backtrace (bt) to see the current context.

Ignore top 4 frames belonging to leak detection library of gdb.

__rtc_event () at ../../../Src/gnu/gdb/infrtc.c:1329
1329 in ../../../Src/gnu/gdb/infrtc.c
(gdb) x/16gx 0x6000000001fc0630 -16
0x6000000001fc0620: 0xfeedfacefeedface 0xfeedfacefeedface
0x6000000001fc0630: 0x000000019fffffff 0xd94013709fffffff
0x6000000001fc0640: 0xd94013709fffffff 0xd94013789fffffff
0x6000000001fc0650: 0xd94013789fffffff 0xd94013809fffffff
0x6000000001fc0660: 0xd94013809fffffff 0xd94013889fffffff
0x6000000001fc0670: 0xd94013889fffffff 0xd94013909fffffff
0x6000000001fc0680: 0xd94013909fffffff 0xd95008409fffffff
0x6000000001fc0690: 0xd95008409fffffff 0xd95008509fffffff
(gdb)
0x6000000001fc06a0: 0xd95008509fffffff 0xd95008609fffffff
0x6000000001fc06b0: 0xd95008609fffffff 0xd95008709fffffff
0x6000000001fc06c0: 0xd95008709fffffff 0xd95008809fffffff
0x6000000001fc06d0: 0xd95008809fffffff 0xd95008909fffffff
0x6000000001fc06e0: 0xd95008909fffffff 0xd95008a09fffffff
0x6000000001fc06f0: 0xd95008a09fffffff 0xd95008b09fffffff
0x6000000001fc0700: 0xd95008b09fffffff 0xd9400fb89fffffff
0x6000000001fc0710: 0xd9400fb89fffffff 0xd9400fc09fffffff
(gdb)
0x6000000001fc0720: 0xd9400fc09fffffff 0xd9400fc89fffffff
>>0x6000000001fc0730: 0xd9400fc850617373 0x776f72643a200020
0x6000000001fc0740: 0x6000000001fc1170 0x6000000001fc0a90
0x6000000001fc0750: 0x6000000001fc06d0 0x0000000000000000
0x6000000001fc0760: 0x6000000001fc1ee0 0x0000000000000000
0x6000000001fc0770: 0x6000000001aa8450 0x000000160000004b
0x6000000001fc0780: 0x6000000001fc0720 0x6000000001fc0770
0x6000000001fc0790: 0x6000000001fc0a90 0x6000000001fc0a90

The line with the leading >> seem to be corrupted by the string "Password: ".

There any way I can watch all the read/write on that kind of allocation?

It seems some dynamic structure allocated in pam_sm_authenticate but I don't have access to this code so it is not really simple to debug

[Dennis Handly]

>I've been able to run my program for a while with check-heap and check-heap string.

I would expect it to be blasted by a string operation. Of course that could be inlined and not show up.

>warning: Memory block (size = 260 address = 0x6000000001fc0630) appears to be corrupted at the end.

Yes. But it seems the whole block is bogus since there seems to be a 8 byte pattern that is half there, and missing 9fffffff. (Unless the pointers have been shifted down 4 bytes: 9fffffffd9401370)

>The line with the leading >> seem to be corrupted by the string "Password: ".

Yes.

> There any way I can watch all the read/write on that kind of allocation?

If you use "help set heap-check" there is something about monitoring addresses but if the address isn't the same, you'll have problems. Otherwise if it is the same address you can put a watchpoint on it.

Who printed this poor stack trace? Where are the hex addresses and function offsets?
#1 pam_sm_authenticate() .../libpam_unix.so.1
#2 pam_authenticate() .../libpam.so.1
#3 PamUnixRegistryImpl_checkPassword() .../libtwspamjni.so

Can you get a better one with "bt"?

You might want to disassembly around pam_sm_authenticate so you can figure out was the last call made. (Does #0 exist?)

If it was strcpy/memcpy that may be the one that moved "Password". If free, the corruption was made sooner.

[Marco Salvi]

Hi Dennes,

> Who printed this poor stack trace? Where are the hex addresses and function offsets?

This is not a real stack trace. I've got this running my program in gdb with

set heap-check on
set heap-check string on

After a while the program execution has been interrupted and I have got the stack trace I have sent you.

I don't have same context available but I remember that bt command shown a stack trace starting form the pam_end call and containing a free() when the gdb stop whit the memory corruption messsage.

I think I need someone that have access to pam_sm_authenticate code in order to understand what is going on there

[Dennis Handly]

>I don't have same context available but I remember that bt command shown a stack trace starting form the pam_end call and containing a free()

That seems reasonable, pam_end is where the memory has to be cleaned up. But it is the code that moves "Password" that has the problem.

>I think I need someone that have access to pam_sm_authenticate code

I'll see if I have access.

[Marco Salvi]

About your previous observation regarding the fact that all that structure look to be corrupted, I think you were right.
I run a single thread program and that structure look completely different:

from info heap
...
8 260 1 0x60000000008b8ed0 pam_sm_authenticate()
...

x/16gx 0x60000000008b8ed0
0x60000000008b8ed0: 0x0000000100000000 0x0000000000000000
0x60000000008b8ee0: 0x0000000000000000 0x0000000000000000
0x60000000008b8ef0: 0x0000000000000000 0x0000000000000000
0x60000000008b8f00: 0x0000000000000000 0x0000000000000000
0x60000000008b8f10: 0x0000000000000000 0x0000000000000000
0x60000000008b8f20: 0x0000000000000000 0x0000000000000000
0x60000000008b8f30: 0x0000000000000000 0x0000000000000000
0x60000000008b8f40: 0x0000000000000000 0x0000000000000000

In the case of corruption this structure is plenty of values like 9ffff.. that look like stack addresses

Anyway I don't know whether or not that means anything.

[Dennis Handly]

>About your previous observation regarding the fact that all that structure look to be corrupted, I think you were right.
>I run a single thread program and that structure look completely different:

At least that 1 is there:
0x60000000008b8ed0: 0x00000001

>Anyway I don't know whether or not that means anything.

Perhaps not.

>I think I need someone that have access to pam_sm_authenticate code

Basically "Password: " is passed to:
__pam_get_i18n_msg(pamh, "pam_unix", 1, 31, "Password: ") (which calls catgets) and the result copied to the stack, with 512 bytes of space.
This local, prompt, is then passed to:
__pam_get_authtok(pamh, PAM_PROMPT, PAM_AUTHTOK, passwd_len, prompt, &password)

In _pam_get_authtok, it is again copied to a local:
strncpy(messages[0], prompt, sizeof (messages[0]));

messages is passed to __pam_get_input then to __pam_input_output (4th parm).

Which passes this to your convert function as msg.

[Marco Salvi]

Hi Dennis,
I've just ask HP support to deal with this problem but I have really appreciated your help on this.
At the moment I've been able to recreate it in more simple program.

Basically I've written a java program that load my shared library with pam code and do some sql query on oracle database.

Running this program in the debugger with

set heap-check on
set heap-check string on

I've got this

[Switching to thread 14 (system thread 229340)]
warning: Memory block (size = 260 address = 0x6000000001387af0) appears to be corrupted at the end.
Allocation context not found

#1 pam_sm_authenticate() from /usr/lib/security/hpux64/libpam_unix.so.1
#2 pam_authenticate() from /usr/lib/hpux64/libpam.so.1
#3 PamUnixRegistryImpl_checkPassword() from /local/tws83/appserver/lib/ext/libtwspamjni.so
#4 Java_com_ibm_tws_pam_security_registry_PamUnixRegistryImpl_pam_1authenticate() from /local/tws83/appserver/lib/ext/libtwspamjni.so
warning: Use command backtrace (bt) to see the current context.

Ignore top 4 frames belonging to leak detection library of gdb.

__rtc_event () at ../../../Src/gnu/gdb/infrtc.c:1329
1329 ../../../Src/gnu/gdb/infrtc.c: No such file or directory.
in ../../../Src/gnu/gdb/infrtc.c
Current language: auto; currently c

The same program without using jdbc driver doesn't report any memory corruption.

I'll try to do some more debug on this but I hope hp support will come to a solution soon

Thanks again for your help and all the debugger tips you have taught me