topic Re: calloc() not returning cleared memory in Operating System - HP-UX

calloc() not returning cleared memory

R Hooper — Tue, 16 Feb 2010 04:33:16 GMT

Good day. First post on this forum, so apologies in advance if I stray from the ideal approach...

My C application has tripped up on some non-cleared memory obtained via "calloc()". First, basic backround:

- Developing on HP-UX 11.31 Itanium, 32-bit app.

- Related "swlist" info:

B9007AA C.11.31.04.2 HP C/aC++ Developer's Bundle
...
PHSS_36354 1.0 assembler patch
PHSS_37959 1.0 LIBCL patch
PHSS_38135 1.0 linker + fdp cumulative patch
PHSS_38527 1.0 Aries cumulative patch
PHSS_39100 1.0 Math Library Cumulative Patch
PHSS_39102 1.0 Integrity Unwind Library
PHSS_40544 1.0 aC++ Runtime (IA: A.06.25, PA: A.03.85)
PHSS_40631 1.0 HP C/aC++ Compiler (A.06.24)
PHSS_40633 1.0 u2comp/be/plugin (C.06.24)

Frustratingly, I'm not able to reproduce the problem with a slimline, standalone .c build - it seems to have a lot to do with the size/structure of the entire application. However, I *can* reproduce it with simple test code in main:

TEST #1:

int main(int argc, char **argv)
{
int *test_p;

test_p = calloc( 6, sizeof(int) );

if ( test_p )
{
printf("Allocated address %08.8lX: %X %X %X %X %X %X\n",
test_p,
test_p[0],
test_p[1],
test_p[2],
test_p[3],
test_p[4],
test_p[5] );

free( test_p );
}
else
{
printf("Some text.\n" );
}

return 0;
}

Output:

Allocated address 40165FF0: 0 0 40165FDC 1 0 0

The "... 40165FDC 1 ..." shouldn't be there at all.

As mentioned previously, the problem has a lot to do with overall process structure/sizing. For example, changing the text size of the second printf (not actually executed) results in different memory contents from "calloc()":

TEST #2: Different text size

int main(int argc, char **argv)
{
int *test_p;

test_p = calloc( 6, sizeof(int) );

if ( test_p )
{
printf("Allocated address %08.8lX: %X %X %X %X %X %X\n",
test_p,
test_p[0],
test_p[1],
test_p[2],
test_p[3],
test_p[4],
test_p[5] );

free( test_p );
}
else
{
printf("Some text.1234567890123456\n" );
}

return 0;
}

Output:

Allocated address 40166000: 0 0 0 0 0 0

I've poked at this a number of ways, but no blinding flash of revelation has occurred, I'm afraid...

- I ensured the latest HP compiler patches were applied.

- I used gdb's heap-check options, but calloc() produced "clean" memory under gdb. The same binary produced "dirty" memory when run outside gdb.

- I compiled/linked with "+check=all" and confirmed from the resulting report that the allocated area was indeed 6x4=24 bytes.

- It seems that adding or removing 16 characters from any text string is enough to "move" the problem. As a rule, if I can re-jig text or static structures elsewhere in the app to line up the "calloc()" address at or about 0x40165FF0, the returned memory is dirty.

- Our default application makefile uses "-Ae" compiler flag. I tried messing about with settings like "+u1" and "+Olit=none" to play with alignment and string/constant storage, but - unsurprisingly - the dirty memory issue was still reproducible.

I realise there is a total absence of test code capable of reproducing this outside my environment. Apologies... I wish there was. But to be honest, it's hard to reproduce even here - simply editing some text string data is enough to shift the problem sideways.

I've dealt with my fair share of memory corruption strangeness over the years (good old C), but this one seems to lead to an area outside my control. It looks like memory heap strangeness, but I don't know why I'm experiencing it... and why it hasn't been seen by others.

I'd certainly appreciate any thoughts on this.

Thanks in advance,

Richard Hooper

Re: calloc() not returning cleared memory

Dennis Handly — Tue, 16 Feb 2010 15:46:26 GMT

>some non-cleared memory obtained via calloc().

That's pretty hard to do. Do you have a threaded application? What opt level do you use?
I assume you have included and ?

>I can reproduce it with simple test code in main:

Are you saying that your above example won't fail by itself and you need more of your application "fluff" to get it to fail?

>I'd certainly appreciate any thoughts on this.

Put a hardware watchpoint on the address in question. You should see it being set to 0. Then by the bad guy corrupting it.
(gdb) watch *(int*)(0x40165FF0+2*4)
(Where the above hex address is what gets returned by calloc.)

Re: calloc() not returning cleared memory

R Hooper — Wed, 17 Feb 2010 01:31:20 GMT

Thanks for the assistance.

> Do you have a threaded application?
> What opt level do you use?

No threading, and no specific optimisation flags. This is 32-bit app, using "-Ae".

> I assume you have included and ?

Yes indeed.

> >I can reproduce it with simple test code in main:
> Are you saying that your above example won't fail by itself and you need more of your application "fluff" to get it to fail?

Correct... even though the fluff isn't executed.

> Put a hardware watchpoint on the address in question. You should see it being set to 0.
> Then by the bad guy corrupting it.
> (gdb) watch *(int*)(0x40165FF0+2*4)
> (Where the above hex address is what gets returned by calloc.)

Done. Here are the results:

$ gdb calloc_test
HP gdb 5.9 for HP Itanium (32 or 64 bit) and target HP-UX 11.2x.
Copyright 1986 - 2001 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 5.9 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.
..
(gdb) break main
Breakpoint 1 at 0x401b530:0: file test.c, line 135 from /home/test/bin/calloc_test.
(gdb) run
Starting program: /home/test/bin/calloc_test

Breakpoint 1, main () at test.c:135
135 test_p = calloc( 6, sizeof(int) );
(gdb) watch *(int*)(0x40165FF0+2*4)
Cannot evaluate expression, placing a deferred watchpoint.
Watchpoint 2: *(int *) (1075208176 + 2 * 4)
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 0
New value = 1075207245
0x200000007e8c86b0:1 in grow_arena+0x5f1 () from /usr/lib/hpux32/libc.so.1
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 1075207245
New value = 1075208157
0x200000007e8bf350:1 in real_malloc+0x8f1 () from /usr/lib/hpux32/libc.so.1
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 1075208157
New value = 1075208156
0x200000007e8c8d20:2 in grow_arena+0xc62 () from /usr/lib/hpux32/libc.so.1
(gdb) c
Continuing.
Allocated address 40165FF0: 0 0 40165FDC 1 0 0

Program exited normally.
(gdb) quit
$

Re: calloc() not returning cleared memory

Dennis Handly — Wed, 17 Feb 2010 13:39:02 GMT

>Here are the results:

Sorry, I should have mentioned getting a stack trace (bt) after every change.

>in grow_arena+0x5f1 libc.so.1

A stack trace here and below, will tell us whether it is in calloc or printf that is overwriting the region.

Re: calloc() not returning cleared memory

David Johns — Wed, 17 Feb 2010 14:51:36 GMT

Hi Richard:

On HPUX B.11.31 U ia64, cc: HP C/aC++ B3910B A.06.20 [May 13 2008], with the -Ae flag I get:

Allocated address 400123F0: 0 0 0 0 0 0

It doesn't look like a compiler issue.

On my Mac with gcc:

$ gcc -g -Wall -pedantic test_p.c -o test_p
test_p.c: In function 'main':
test_p.c:18: warning: '0' flag ignored with precision and '%X' printf format
test_p.c:18: warning: format '%08.8lX' expects type 'long unsigned int', but argument 2 has type 'int *'
$ ./test_p
Allocated address 003000F0: 0 0 0 0 0 0

Do you think the warning has any bearing on the matter?

Very strange output indeed.

Cheers,
-Dave

Re: calloc() not returning cleared memory

Dennis Handly — Wed, 17 Feb 2010 15:05:05 GMT

>David: It doesn't look like a compiler issue.

It isn't obvious, yet. :-)

>Do you think the warning has any bearing on the matter?

No. But as you said, the correct format for pointers is %p.

Re: calloc() not returning cleared memory

R Hooper — Thu, 18 Feb 2010 01:42:58 GMT

Test main() duly fixed with regard to %p.

Backtrace performed at every stage. The output...

$ gdb calloc_test
HP gdb 5.9 for HP Itanium (32 or 64 bit) and target HP-UX 11.2x.
Copyright 1986 - 2001 Free Software Foundation, Inc.
Hewlett-Packard Wildebeest 5.9 (based on GDB) is covered by the
GNU General Public License. Type "show copying" to see the conditions to
change it and/or distribute copies. Type "show warranty" for warranty/support.
..
(gdb) break main
Breakpoint 1 at 0x401b590:0: file test.c, line 136 from /home/test/bin/calloc_test.
(gdb) run
Starting program: /home/test/bin/calloc_test

Breakpoint 1, main () at test.c:136
136 test_p = calloc( 6, sizeof(int) );
(gdb) watch *(int*)(0x40165FF0+2*4)
Cannot evaluate expression, placing a deferred watchpoint.
Watchpoint 2: *(int *) (1075208176 + 2 * 4)
(gdb) bt
#0 main () at test.c:136
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 0
New value = 1075207245
0x200000007e8c86b0:1 in grow_arena+0x5f1 () from /usr/lib/hpux32/libc.so.1
(gdb) bt
#0 0x200000007e8c86b0:1 in grow_arena+0x5f1 () from /usr/lib/hpux32/libc.so.1
#1 0x200000007e8bed20:0 in real_malloc+0x2c0 () from /usr/lib/hpux32/libc.so.1
#2 0x200000007e8bfab0:0 in real_malloc+0x1050 ()
from /usr/lib/hpux32/libc.so.1
#3 0x200000007e8be1c0:0 in _malloc+0x140 () from /usr/lib/hpux32/libc.so.1
#4 0x200000007e77c360:0 in malloc+0x160 () from /usr/lib/hpux32/libc.so.1
#5 0x200000007e7775c0:0 in calloc+0x160 () from /usr/lib/hpux32/libc.so.1
#6 0x401b5d0:0 in main () at test.c:136
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 1075207245
New value = 1075208157
0x200000007e8bf350:1 in real_malloc+0x8f1 () from /usr/lib/hpux32/libc.so.1
(gdb) bt
#0 0x200000007e8bf350:1 in real_malloc+0x8f1 () from /usr/lib/hpux32/libc.so.1
#1 0x200000007e8bfab0:0 in real_malloc+0x1050 ()
from /usr/lib/hpux32/libc.so.1
#2 0x200000007e8be1c0:0 in _malloc+0x140 () from /usr/lib/hpux32/libc.so.1
#3 0x200000007e77c360:0 in malloc+0x160 () from /usr/lib/hpux32/libc.so.1
#4 0x200000007e7775c0:0 in calloc+0x160 () from /usr/lib/hpux32/libc.so.1
#5 0x401b5d0:0 in main () at test.c:136
(gdb) c
Continuing.
Hardware watchpoint 2: *(int *) (1075208176 + 2 * 4)

Old value = 1075208157
New value = 1075208156
0x200000007e8c8d20:2 in grow_arena+0xc62 () from /usr/lib/hpux32/libc.so.1
(gdb) bt
#0 0x200000007e8c8d20:2 in grow_arena+0xc62 () from /usr/lib/hpux32/libc.so.1
#1 0x200000007e8bed20:0 in real_malloc+0x2c0 () from /usr/lib/hpux32/libc.so.1
#2 0x200000007e8bf910:0 in real_malloc+0xeb0 () from /usr/lib/hpux32/libc.so.1
#3 0x200000007e8be1c0:0 in _malloc+0x140 () from /usr/lib/hpux32/libc.so.1
#4 0x200000007e77c360:0 in malloc+0x160 () from /usr/lib/hpux32/libc.so.1
#5 0x200000007e7775c0:0 in calloc+0x160 () from /usr/lib/hpux32/libc.so.1
#6 0x401b5d0:0 in main () at test.c:136
(gdb) c
Continuing.
Allocated address 40165ff0: 0 0 40165FDC 1 0 0

Program exited normally.
(gdb)

Re: calloc() not returning cleared memory

David Johns — Thu, 18 Feb 2010 03:16:33 GMT

Richard:

Have you tried using the +Orecovery compiler option? Which is the default... So maybe +Onorecovery?

After rereading posts I see I'm using an older version of the compiler. It's disquieting to see heap corruption like this in the latest and greatest HP product. Unless you have a hardware problem, this could be a big bug somewhere in the toolchain.

Good luck!
-Dave

Re: calloc() not returning cleared memory

R Hooper — Thu, 18 Feb 2010 03:43:12 GMT

Hi David.

Yes, it's a little unnerving. I'll poke at that recovery switch when I get the chance.

Personally, I wouldn't be putting any money on seeing this reproduced by other parties... It's hard to reproduce it myself! I have to add or remove lines of code or static data from my app until the calloc'd area hits the magic address (around 0x40165FF0).

I'm not saying the address in question is a factor, but it's the only reliable guide I have when playing with the issue. Maybe my app could be at the cusp of crossing some meaningful address/sizing boundary, and that's revealed a bug in the dynamic memory heap management. Apologies for the diversion into pure conjecture! :)

Richard

Re: calloc() not returning cleared memory

Dennis Handly — Thu, 18 Feb 2010 09:08:33 GMT

Are you linking with libpthread?

>Backtrace performed at every stage

They indicate that it never zeroes out the region.

You may have QXCR1000805698 where it tries to optimize and not clear the region because it knows the kernel does this for new pages. This should be fixed in PHCO_39526.

>Maybe my app could be at the cusp of crossing some meaningful address/sizing boundary

Yes, the brk boundary.

>Dave: Have you tried using the +Orecovery compiler option?

Not sure why you think this option may help??

>It's disquieting to see heap corruption

It's a heap optimization gone terribly terribly wrong. ;-)

Re: calloc() not returning cleared memory

R Hooper — Fri, 19 Feb 2010 00:25:18 GMT

Sounds very promising indeed. This is a demo box from HP - I guess we need to give it some extra TLC!

I'll schedule up any outstanding patches for the weekend, and will re-test on Monday.

Thanks for the detective work - I'll report the results as soon as they are in.

Richard

Re: calloc() not returning cleared memory

R Hooper — Mon, 22 Feb 2010 03:30:34 GMT

PHCO_39526 nailed it. A glorious sight...

./calloc_test
Allocated address 40165ff0: 0 0 0 0 0 0

Thanks very much!

Richard