malloc failed while growing stack (ksh script)

[Oren Hollander]

Hi, while executing a chain of ksh scripts core is dumped
by the 1st script (guess on behalf of ksh?) - memory fault. Some neat strings in the core file: malloc failed while growing stack
eE;,TtFf%
ksh: sysconf error
ksh: no memory
ksh: no memory
Usually it stops in lines such
echo $1 | tr -A '[a-z]' '[A-Z]' or echo `echo $cmd cut -f 1 -d " "`
ksh version: Version M-11/16/88 on Tru64 5.1A.
These scripts ran with no problems on our previous OS -Tru64 4.0G. Don't know if the same ksh was used. Been told that kernel parameters were retained (so user's limits).
One batch uses as a test case: avg. failure ratio: ~1/20.
tried to: 1.increase avail. stack size for the proc - no effect. 2. use explicit path when calling the above commands (/usr/bin/*). No cores since (after ~185 runs of the test batch). Can it be something with the interaction between shell builtins to non-builtins ?
read about such a thing on HP-UX. Can it be due to command search elimination ?
Thanks

[A. Clay Stephenson]

I haven't worked with Digital UNIX in about 4 years but you should look at the kernel parameter maxdsiz. It's a little misleading to talk about growing a stack which is limited by maxssiz but malloc is limited by maxdsiz.

You might also look for a tunable something like ncargmax or large_ncargs that limits the size of the argument list that the exec() system call will handle.

[Oren Hollander]

Thanks.
The settings are:
per_proc_data_size = 8589924592
max_per_proc_data_size = 8589924592

[Oren Hollander]

Debug attempts: as it's impossible to open the shell script & the core with debugger, i tried to do open the core & ksh. ksh is stripped, so there's nothing much to look at. using gdb -> bt -
#0 0x12001ebe0 in mac_here ()
#1 0x12001e32c in mac_expand ()

Couldn't find much about these functions - help with that would be appreciated.