topic malloc error in Operating System - HP-UX

malloc error

David Gwin_1 — Wed, 08 Jun 2005 14:21:11 GMT

Some assistance please,

K260, 512mb ram, HPUX 10.20

I just attempted to restore a file set when I got the malloc error:

Starting file recovery...
frecover(5435): malloc error while attempting to read index
frecover(4301): /tmp/datafilename not recovered from backup media

I know enough to check the maxdsiz parm but that is where the problem really starts. maxdsiz is current 2063835136 (0x7B03A000) and if I attempt to increase I get the error that I have exceeded the maximum size for maxdsiz. I can recover the index file and it is rather large too, 74449920 (there are thousands of files).

So my question is, How can I restore my file if I can increase maxdsiz? Can I recover without the index file?

Re: malloc error

TwoProc — Wed, 08 Jun 2005 14:29:49 GMT

Well, I don't think it has anything to do with maxdsize because your machine is at 1/2 gig, your maxdsize is 2G and you're trying to allocate 75meg. I just think you've used up all your free ram. I think you could make a larger swap area and let all of that memory allocation run from there. It would be slow, but you might be able to recover your file.

Re: malloc error

harry d brown jr — Wed, 08 Jun 2005 14:35:25 GMT

(1) get new hardware (your hardware is obsolete)
(2) Install 11.11 (or better: 11.23) (your OS is obsolete)

live free or die
harry d brown jr

Re: malloc error

D Block 2 — Wed, 08 Jun 2005 14:35:27 GMT

what is your output from: ulimit ?

Re: malloc error

A. Clay Stephenson — Wed, 08 Jun 2005 14:40:08 GMT

Maxdsiz is already more than adequate and there is really no point in it being more than 1GB (and even that is ludicrous given your amount of physical memory). You are problem running out of virtual address space because you don't have enough swapspace. Malloc can also fail if ulimit has been applied.

Do a ulimit -Hd and a ulimit -Sd.

In any event, you only need enough shared memory to hold the index.

Re: malloc error

David Gwin_1 — Wed, 08 Jun 2005 14:59:52 GMT

I would love to get new hardware/software however that is not an option.

ulimit -Hd
2015464

and the same for -Sd

swapinfo
Kb Kb Kb PCT
TYPE AVAIL USED FREE USED
dev 524288 13216 511072 3%
reserve - 151540 -151540
memory 389120 256508 132612 66%

Re: malloc error

David Gwin_1 — Wed, 08 Jun 2005 15:10:49 GMT

swapinfo table didn't display too well,

_____________Kb______Kb______Kb___PCT_
TYPE______AVAIL____USED____FREE__USED_
dev_______524288___13216__511072____3%_
reserve_________-__151540__-151540
memory___389120__256508___132612___66%

Re: malloc error

A. Clay Stephenson — Wed, 08 Jun 2005 15:15:48 GMT

This is 32-bit HP-UX so both the stack and the data segment are allocated from the same 1GB quadrant. This means, for example, if you had set maxssiz to 800MB then at most only 200MB would actually be available for the data segment regardless of maxdsiz. (and actually about 40MB less than that). Check you setting of maxssiz. Anything bigger than 32MB is only needed for very badky written code.

Re: malloc error

David Gwin_1 — Wed, 08 Jun 2005 16:11:14 GMT

Thank you for the information.

maxssiz is 83570688 (0x04FB3000)

Re: malloc error

David Gwin_1 — Thu, 09 Jun 2005 14:43:32 GMT

I have made a few adjustments but I'm still not having any luck finding that magic answer.

Other than upgrading the HW or OS, are there any other suggestions as to how I might go about restoring this data tape. Any other parms that could be adjusted up or down?

I can see that I need to change my backup process but that's not going to help with the files that are already on backup tapes.

Re: malloc error

Ermin Borovac — Thu, 09 Jun 2005 20:06:06 GMT

Maybe you can try adding another swap device if have spare disk space.

If you have 'glance' installed on your system you can watch RSS/VSS (resident and virtual memory size) in process detail window (press s and select frecover PID) to see how much memory frecover is trying to allocate.

Alternatively you can run trace frecover with tusc.

# tusc -o tusc.log -fv -s brk frecover ...

You will see see number of brk() calls as frecover tries to increase the heap.

brk(0x400021e8) .......................................... = 0
brk(0x400041d0) .......................................... = 0
brk(0x40005000) .......................................... = 0
brk(0x40007000) .......................................... = 0
brk(0x40008000) .......................................... = 0
brk(0x4001e000) .......................................... = 0
brk(0x40022000) .......................................... = 0

In this example traced program successfully allocated 0x22000 bytes (139264 in decimal).

In your case you will see failed brk() call at the end when frecover fails. From there you can find out how much memory it's trying to allocate.

Re: malloc error

A. Clay Stephenson — Thu, 09 Jun 2005 21:20:52 GMT

At this point, I'm going to ask that you humor me and reduce maxdsiz to about 512MB, maxssiz to about 64MB (or 32MB) and maxtsiz to about 128MB. Since you already have 1X memory as swapspace, swapmem_on=1 is doing nothing for you so you might as well disable pseudoswap. When you reboot under the new kernel make sure that no other non-essential tasks are running. Given your machine's configuration, your present values are a little ridiculous so returning them to more or less sane levels couldn't hurt. I would also limit buffer cache to about 60MB by setting bufpages to 15360 and make sure nbuf is left at zero. This will turn off dynamic buffer cache.

At the very least, these values will make any errors more understandable.

Re: malloc error

David Gwin_1 — Fri, 10 Jun 2005 11:19:22 GMT

I don't have glance or tusc installed on this system. I have the very basic system with the last and final hw/sw patches installed for 10.20.

Because this is a heavily used system, I am not able to shut it down during the day. I'll give the reduced maxdsiz, maxssiz and maxtsiz a try this weekend.

Could I use dd to dump the tape contents to another drive? What format is fbackup? Or is its own format?

Re: malloc error

A. Clay Stephenson — Fri, 10 Jun 2005 11:29:12 GMT

Dd is useless to recover this tape because it uses tape marks and variable blocking. You could use frecover on another HP-UX box but I think as soon as you can reduce the load on this box, you should be able to run frecover.

Re: malloc error

harry d brown jr — Fri, 10 Jun 2005 13:07:53 GMT

Auditers would kill us if we still had 10.20 running!

hopefully you are a non-profit!

live free or die
harry d brown jr

Re: malloc error

Ermin Borovac — Sat, 11 Jun 2005 00:02:03 GMT

Even if you don't have glance you can still run 'top' and watch SIZE and RES for frecover before it fails. I think it's important to find out amount of memory it's trying to allocate before making any changes. It also wouldn't hurt to capture 'Memory' line from 'top' and 'swapinfo -tam' before frecover failure.

PS Forgot this was 10.20 system, you wouldn't be able to run tusc anyway!

Re: malloc error

D Block 2 — Sat, 11 Jun 2005 21:10:50 GMT

vmstat -n 5 100

I'm thinking vmstat should be around.. on 10.20

also, you can use 'adb' to print the 'freemem' in a loop, to just look at free memory in the kernel.

Re: malloc error

David Gwin_1 — Fri, 17 Jun 2005 09:15:49 GMT

Just to give a bit more information,

This past weekend, made a number of attempts to restore the data tape. Changed parms (maxdsiz, maxssiz, maxtsiz, and others), watched the memory (the best that I could), reboots, single user mode, and anything else that i could think of and still no success.

I found someone here locally who has a D-box HP system w/ 11.x 2gb of memory. STILL no luck at restoring the tape.

Any other ideas?

Re: malloc error

A. Clay Stephenson — Fri, 17 Jun 2005 09:39:06 GMT

This really sounds as though you have a corrupt fbackup image on the tape. It sounds as though reading back your backup tapes was not a regular practice.

Re: malloc error

David Gwin_1 — Fri, 17 Jun 2005 15:13:19 GMT

Does any one know if there is a limit to the size (space allowed) for the index header file on a tape. IE the first 64mb is reserved for the index header???
I just check the last know good backup is 46mb and there are 760,000 lines in the index file. The file is looks normal:
# 1 /path/filename

The backup that is failing is 74mb and 930,000 lines in the index file. This index is a bit different:
832 1 /path/filename

Each line has what looks like a filesize followed by about 20 spaces.