broken tail? tail -559 works. tail -561 fails.

[Steve Post]

I have a simple text log file.
I want to truncate the log.

logfile is 850 lines long.
cat logfile | tail -559 > logfile.tmp
logfile.tmp should be 559 lines long. And it is.

cat logfile | tail -561 > logfile.tmp
logfile.tmp should be 561 lines long. It is still only 559 lines.

cat logfile | tail -2000 > logfile.tmp
logfile.tmp should be 850 lines long (i.e. unmodified). Yet it has been truncated to 559 lines.

cat logfile | se -e :a -e '$q:N,2001,$D;ba' > logfile.tmp.
This logfile.tmp is correct. But I don't understand the strange sed syntax. I just pulled it off the net. I don't trust it because I don't understand it.

So my question:
Any idea why "tail -559 works" and "tail -561" fails? It this is some type of memory limitation, why does that strange sed command work?

K570 HP9000 HPUX11.0

steve

[A. Clay Stephenson · Accepted solution]

Depending upon the flavor of UNIX there is a fixed-size buffer for tail operations relative to EOF. Typically it's a 20K buffer.

If you are uncomfortable with the sed command then craft a Perl or awk equivalent.

[Pete Randall]

Steve,

From the man page:

"Tails relative to end-of-file are stored in a 20-Kbyte buffer, and thus are limited in length."


Pete

[Steve Post]

Thanks. That makes sense.

I ran
tail -n 580 logfile > f1
tail -n 590 logfile > f2
tail -n 200000 logfile > f3
ls -l f?

I see the file size for files f1, f2 and f3 are ALL 20468 bytes.

#LEN=total lines of logfile

#TAILN=the total I want it to be

#L=the starting line for the awk.

LEN=`wc -l logfile | cut -f1 -d\ `
TAILN=800
L=`expr $LEN - $TAILN`
if [ $L -le 0 ] ; then
cat logfile
else
cat logfile | \
awk -v L=$L '// { if (NR < L) print ;}' > f4

File f4 is 28420 bytes, and 800 lines long.

Steve

[Murali K Sethuraman]

The solution would be to do make tail work from the top of the file rather than the end of file, which is the default. You could use:
tail -n +

With "tail -n <# lines> ", the tail would break on any # of lines, depending on the size of the file and the number of lines you are trying to do tail on. "559" is in case of your file with 850 lines. I encountered the problem at 1024 lines for a file with 2000+ lines, 210 lines for a file with 550+ lines etc.

[Pat Obrien_1]

In 11i v1 tail is broke. PHCO_27138 will fix the broken tail command.