Memory Fault (coredump) when applying sed to large file

[Bruce Mulder]

I have a sed script which I'm using to pre-format a text file to work around the 3000 character max line length limitation of awk. The sed script works well until the input file gets to be larger than 100Mb. The sed script will report a memory fault and will coredump on these large files.

Any suggestions on how to:
1) prevent or diagnose the sed coredumps
2) work arounds for the awk line length limit

TIA

[Sridhar Bhaskarla]

Hi,

This could be due to maxdsiz kernel parameter. Check the value of your maxdsiz on the system and adjust it accordingly.

-Sri

[S.K. Chan]

sed should be able to take 4000 chars.
If your script has comments of more than 200 chars, awk will coredumps !! This might be fixed already in 11.x. I'm not sure ..

[A. Clay Stephenson · Accepted solution]

Hi Bruce:

Your fix for the awk line length limit is to use gawk available from any of the HP-UX Porting Centres.

http://hpux.connect.org.uk/hppd/hpux/Gnu/gawk-3.1.0/

Unlike awk, gawk has dynamically allocated record lengths and can be set from the command line to any arbitrary length.

Clay

[Bruce Mulder]

Thanks for your replys!

I'll check these out with my SA and let you know.

[Bruce Mulder]

Thanks for your suggestions, however, the Configuration Manangment team here would rather that the system be written in PERL and has rejected my request for gawk.

Bottomline, I may never know if gawk was truly the answer, tho I suspect it is.