Languages and Scripting
Showing results for 
Search instead for 
Do you mean 

Script to work on a large file

Frequent Advisor

Script to work on a large file

I need a script to remove header lines of a printing file. For example, I know it started with a identifier GATB, and from the line containing this GATB, I should remove 9 lines. The document has 800+ pages, could you help me to get this script done?

Thanks in advance,
Chushia
3 REPLIES
Occasional Advisor

Re: Script to work on a large file


This should do it:

COUNT=0

while read LINE
do
echo $LINE | grep GATB > /dev/null
if [ $? = 0 ] ; then
COUNT=1
fi
if [ $COUNT = 0 ] ; then
echo $LINE
else
if [ $COUNT -lt 11 ] ; then
((COUNT=COUNT+1))
else
COUNT=0
echo $LINE
fi
fi
done < input > output
Frequent Advisor

Re: Script to work on a large file

Thanks for the script. It did not work.

I studied the file again, and found a pattern that the header ends with a dashed line -----------------------

so I used sed to remove them by:
sed '/GATB/,/\-\-\-\-\-\-\-\-\-\-\-\-\-\-/d' < input > output

Highlighted
Acclaimed Contributor Acclaimed Contributor

Re: Script to work on a large file

>sed '/GATB/,/\-\-\-\-\-\-\-\-\-\-\-\-\-\-/d' < input > output

You don't really need to escape a "-":
sed '/GATB/,/--------------/d' input > output