calloc() not returning cleared memory

[R Hooper]

Good day. First post on this forum, so apologies in advance if I stray from the ideal approach...

My C application has tripped up on some non-cleared memory obtained via "calloc()". First, basic backround:

- Developing on HP-UX 11.31 Itanium, 32-bit app.

- Related "swlist" info:

B9007AA C.11.31.04.2 HP C/aC++ Developer's Bundle
...
PHSS_36354 1.0 assembler patch
PHSS_37959 1.0 LIBCL patch
PHSS_38135 1.0 linker + fdp cumulative patch
PHSS_38527 1.0 Aries cumulative patch
PHSS_39100 1.0 Math Library Cumulative Patch
PHSS_39102 1.0 Integrity Unwind Library
PHSS_40544 1.0 aC++ Runtime (IA: A.06.25, PA: A.03.85)
PHSS_40631 1.0 HP C/aC++ Compiler (A.06.24)
PHSS_40633 1.0 u2comp/be/plugin (C.06.24)


Frustratingly, I'm not able to reproduce the problem with a slimline, standalone .c build - it seems to have a lot to do with the size/structure of the entire application. However, I *can* reproduce it with simple test code in main:

TEST #1:

int main(int argc, char **argv)
{
int *test_p;

test_p = calloc( 6, sizeof(int) );

if ( test_p )
{
printf("Allocated address %08.8lX: %X %X %X %X %X %X\n",
test_p,
test_p[0],
test_p[1],
test_p[2],
test_p[3],
test_p[4],
test_p[5] );

free( test_p );
}
else
{
printf("Some text.\n" );
}

return 0;
}

Output:

Allocated address 40165FF0: 0 0 40165FDC 1 0 0


The "... 40165FDC 1 ..." shouldn't be there at all.


As mentioned previously, the problem has a lot to do with overall process structure/sizing. For example, changing the text size of the second printf (not actually executed) results in different memory contents from "calloc()":

TEST #2: Different text size

int main(int argc, char **argv)
{
int *test_p;

test_p = calloc( 6, sizeof(int) );

if ( test_p )
{
printf("Allocated address %08.8lX: %X %X %X %X %X %X\n",
test_p,
test_p[0],
test_p[1],
test_p[2],
test_p[3],
test_p[4],
test_p[5] );

free( test_p );
}
else
{
printf("Some text.1234567890123456\n" );
}

return 0;
}

Output:

Allocated address 40166000: 0 0 0 0 0 0


I've poked at this a number of ways, but no blinding flash of revelation has occurred, I'm afraid...

- I ensured the latest HP compiler patches were applied.

- I used gdb's heap-check options, but calloc() produced "clean" memory under gdb. The same binary produced "dirty" memory when run outside gdb.

- I compiled/linked with "+check=all" and confirmed from the resulting report that the allocated area was indeed 6x4=24 bytes.

- It seems that adding or removing 16 characters from any text string is enough to "move" the problem. As a rule, if I can re-jig text or static structures elsewhere in the app to line up the "calloc()" address at or about 0x40165FF0, the returned memory is dirty.

- Our default application makefile uses "-Ae" compiler flag. I tried messing about with settings like "+u1" and "+Olit=none" to play with alignment and string/constant storage, but - unsurprisingly - the dirty memory issue was still reproducible.


I realise there is a total absence of test code capable of reproducing this outside my environment. Apologies... I wish there was. But to be honest, it's hard to reproduce even here - simply editing some text string data is enough to shift the problem sideways.


I've dealt with my fair share of memory corruption strangeness over the years (good old C), but this one seems to lead to an area outside my control. It looks like memory heap strangeness, but I don't know why I'm experiencing it... and why it hasn't been seen by others.


I'd certainly appreciate any thoughts on this.

Thanks in advance,


Richard Hooper

[Dennis Handly]

>some non-cleared memory obtained via calloc().

That's pretty hard to do. Do you have a threaded application? What opt level do you use?
I assume you have included and ?

>I can reproduce it with simple test code in main:

Are you saying that your above example won't fail by itself and you need more of your application "fluff" to get it to fail?

>I'd certainly appreciate any thoughts on this.

Put a hardware watchpoint on the address in question. You should see it being set to 0. Then by the bad guy corrupting it.
(gdb) watch *(int*)(0x40165FF0+2*4)
(Where the above hex address is what gets returned by calloc.)

[R Hooper]

Thanks for the assistance.

> Do you have a threaded application?
> What opt level do you use?

No threading, and no specific optimisation flags. This is 32-bit app, using "-Ae".


> I assume you have included and ?

Yes indeed.


> >I can reproduce it with simple test code in main:
> Are you saying that your above example won't fail by itself and you need more of your application "fluff" to get it to fail?

Correct... even though the fluff isn't executed.

> Put a hardware watchpoint on the address in question. You should see it being set to 0.
> Then by the bad guy corrupting it.
> (gdb) watch *(int*)(0x40165FF0+2*4)
> (Where the above hex address is what gets returned by calloc.)

Done. Here are the results:


$ gdb calloc_test
HP gdb 5.9 for HP Itanium (32 or 64 bit) and target HP-UX 11.2x.
Copyright 1986 - 2001 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 5.9 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.
..
(gdb) break main
Breakpoint 1 at 0x401b530:0: file test.c, line 135 from /home/test/bin/calloc_test.
(gdb) run
Starting program: /home/test/bin/calloc_test

Breakpoint 1, main () at test.c:135
135 test_p = calloc( 6, sizeof(int) );
(gdb) watch *(int*)(0x40165FF0+2*4)
Cannot evaluate expression, placing a deferred watchpoint.
Watchpoint 2: *(int *) (1075208176 + 2 * 4)
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 0
New value = 1075207245
0x200000007e8c86b0:1 in grow_arena+0x5f1 () from /usr/lib/hpux32/libc.so.1
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 1075207245
New value = 1075208157
0x200000007e8bf350:1 in real_malloc+0x8f1 () from /usr/lib/hpux32/libc.so.1
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 1075208157
New value = 1075208156
0x200000007e8c8d20:2 in grow_arena+0xc62 () from /usr/lib/hpux32/libc.so.1
(gdb) c
Continuing.
Allocated address 40165FF0: 0 0 40165FDC 1 0 0

Program exited normally.
(gdb) quit
$

[Dennis Handly]

>Here are the results:

Sorry, I should have mentioned getting a stack trace (bt) after every change.

>in grow_arena+0x5f1 libc.so.1

A stack trace here and below, will tell us whether it is in calloc or printf that is overwriting the region.

[David Johns]

Hi Richard:


On HPUX B.11.31 U ia64, cc: HP C/aC++ B3910B A.06.20 [May 13 2008], with the -Ae flag I get:

Allocated address 400123F0: 0 0 0 0 0 0

It doesn't look like a compiler issue.

On my Mac with gcc:

$ gcc -g -Wall -pedantic test_p.c -o test_p
test_p.c: In function 'main':
test_p.c:18: warning: '0' flag ignored with precision and '%X' printf format
test_p.c:18: warning: format '%08.8lX' expects type 'long unsigned int', but argument 2 has type 'int *'
$ ./test_p
Allocated address 003000F0: 0 0 0 0 0 0

Do you think the warning has any bearing on the matter?

Very strange output indeed.

Cheers,
-Dave

[Dennis Handly]

>David: It doesn't look like a compiler issue.

It isn't obvious, yet. :-)

>Do you think the warning has any bearing on the matter?

No. But as you said, the correct format for pointers is %p.

[R Hooper]

Test main() duly fixed with regard to %p.

Backtrace performed at every stage. The output...


$ gdb calloc_test
HP gdb 5.9 for HP Itanium (32 or 64 bit) and target HP-UX 11.2x.
Copyright 1986 - 2001 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 5.9 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.
..
(gdb) break main
Breakpoint 1 at 0x401b590:0: file test.c, line 136 from /home/test/bin/calloc_test.
(gdb) run
Starting program: /home/test/bin/calloc_test

Breakpoint 1, main () at test.c:136
136 test_p = calloc( 6, sizeof(int) );
(gdb) watch *(int*)(0x40165FF0+2*4)
Cannot evaluate expression, placing a deferred watchpoint.
Watchpoint 2: *(int *) (1075208176 + 2 * 4)
(gdb) bt
#0 main () at test.c:136
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 0
New value = 1075207245
0x200000007e8c86b0:1 in grow_arena+0x5f1 () from /usr/lib/hpux32/libc.so.1
(gdb) bt
#0 0x200000007e8c86b0:1 in grow_arena+0x5f1 () from /usr/lib/hpux32/libc.so.1
#1 0x200000007e8bed20:0 in real_malloc+0x2c0 () from /usr/lib/hpux32/libc.so.1
#2 0x200000007e8bfab0:0 in real_malloc+0x1050 ()
from /usr/lib/hpux32/libc.so.1
#3 0x200000007e8be1c0:0 in _malloc+0x140 () from /usr/lib/hpux32/libc.so.1
#4 0x200000007e77c360:0 in malloc+0x160 () from /usr/lib/hpux32/libc.so.1
#5 0x200000007e7775c0:0 in calloc+0x160 () from /usr/lib/hpux32/libc.so.1
#6 0x401b5d0:0 in main () at test.c:136
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 1075207245
New value = 1075208157
0x200000007e8bf350:1 in real_malloc+0x8f1 () from /usr/lib/hpux32/libc.so.1
(gdb) bt
#0 0x200000007e8bf350:1 in real_malloc+0x8f1 () from /usr/lib/hpux32/libc.so.1
#1 0x200000007e8bfab0:0 in real_malloc+0x1050 ()
from /usr/lib/hpux32/libc.so.1
#2 0x200000007e8be1c0:0 in _malloc+0x140 () from /usr/lib/hpux32/libc.so.1
#3 0x200000007e77c360:0 in malloc+0x160 () from /usr/lib/hpux32/libc.so.1
#4 0x200000007e7775c0:0 in calloc+0x160 () from /usr/lib/hpux32/libc.so.1
#5 0x401b5d0:0 in main () at test.c:136
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 1075208157
New value = 1075208156
0x200000007e8c8d20:2 in grow_arena+0xc62 () from /usr/lib/hpux32/libc.so.1
(gdb) bt
#0 0x200000007e8c8d20:2 in grow_arena+0xc62 () from /usr/lib/hpux32/libc.so.1
#1 0x200000007e8bed20:0 in real_malloc+0x2c0 () from /usr/lib/hpux32/libc.so.1
#2 0x200000007e8bf910:0 in real_malloc+0xeb0 () from /usr/lib/hpux32/libc.so.1
#3 0x200000007e8be1c0:0 in _malloc+0x140 () from /usr/lib/hpux32/libc.so.1
#4 0x200000007e77c360:0 in malloc+0x160 () from /usr/lib/hpux32/libc.so.1
#5 0x200000007e7775c0:0 in calloc+0x160 () from /usr/lib/hpux32/libc.so.1
#6 0x401b5d0:0 in main () at test.c:136
(gdb) c
Continuing.
Allocated address 40165ff0: 0 0 40165FDC 1 0 0

Program exited normally.
(gdb)

[David Johns]

Richard:

Have you tried using the +Orecovery compiler option? Which is the default... So maybe +Onorecovery?

After rereading posts I see I'm using an older version of the compiler. It's disquieting to see heap corruption like this in the latest and greatest HP product. Unless you have a hardware problem, this could be a big bug somewhere in the toolchain.

Good luck!
-Dave

[R Hooper]

Hi David.

Yes, it's a little unnerving. I'll poke at that recovery switch when I get the chance.

Personally, I wouldn't be putting any money on seeing this reproduced by other parties... It's hard to reproduce it myself! I have to add or remove lines of code or static data from my app until the calloc'd area hits the magic address (around 0x40165FF0).

I'm not saying the address in question is a factor, but it's the only reliable guide I have when playing with the issue. Maybe my app could be at the cusp of crossing some meaningful address/sizing boundary, and that's revealed a bug in the dynamic memory heap management. Apologies for the diversion into pure conjecture! :)

Richard

[Dennis Handly · Accepted solution]

Are you linking with libpthread?

>Backtrace performed at every stage

They indicate that it never zeroes out the region.

You may have QXCR1000805698 where it tries to optimize and not clear the region because it knows the kernel does this for new pages. This should be fixed in PHCO_39526.

>Maybe my app could be at the cusp of crossing some meaningful address/sizing boundary

Yes, the brk boundary.

>Dave: Have you tried using the +Orecovery compiler option?

Not sure why you think this option may help??

>It's disquieting to see heap corruption

It's a heap optimization gone terribly terribly wrong. ;-)