HPUX 11.0: ...Idenifying Files That Grow

[LG Porter]

I have several K580 servers that have HPUX 11.00 installed. Several of the file systems continue to grow such that I'm daily checking to idenify those files that continue to grow each day. Other that using the du command or the find command, is there a method or series of commands that could be scripted togther in order to identify larges files that grow?

[John Dvorchak]

I am not sure why you don't want to use du but you can put this in a script that will identify the biggest files and directories and sort them from largest to smallest. This is the one liner that I use after getting it from Bill Hassel

du -k . | sort -rn | more

I suppose you could script this and write the output to a file for future reference:

du -k . | sort -rn > /tmp/bigfiles

[Sridhar Bhaskarla]

Hi,

I would say you start monitoring your filesystem usage. Once you are notified with the filesystems that exceeded certain threshold, you can run "find" command only in that filesystem to determine the files that are growing. Look at man page of find. You will be interested in "size" option of it.

Below is a sample script that can send you an email if it finds any of the filesystems above 75%.

THRESHOLD=75
EMAIL="yourid@yourdomain.com"
for fs in $(df -F vxfs |awk '{print $1}')
do
USED=$(df -F vxfs -k $fs |awk '/allocation used/ {print $1}')
if [ $USED -ge $THRESHOLD ]
then
echo $fs $USED >> /tmp/tmp$$
fi
done
if [ $(cat /tmp/tmp$$|wc -l) -ge 1 ]
then
mailx -s "Filesystems above 75%" $EMAIL < /tmp/tmp$$
fi

-Sri

[Steven E. Protter · Accepted solution]

I had to revise the prior script to get it to work when no filesystems exceed the threashold.

THRESHOLD=85
EMAIL="yourid@yourdomain.com"
trigger=0
for fs in $(df -F vxfs |awk '{print $1}')
do
USED=$(df -F vxfs -k $fs |awk '/allocation used/ {print $1}')
# echo "post 1"
if [ $USED -ge $THRESHOLD ]
then
(( trigger = trigger + 1 ))
echo $fs $USED >> /tmp/tmp$$
fi
done
# echo "post 2 $trigger"

if [ $trigger -le 1 ]
then
echo "no overfull fs"
exit 0
fi

if [ $(cat /tmp/tmp$$|wc -l) -ge 1 ]
then
mailx -s "Filesystems above ${THREASHOLD}%" $EMAIL < /tmp/tmp$$
fi

Further revision will be required to run it in cron. PATH variable or full path of commands used.

Great script though.

SEP

[Sridhar Bhaskarla]

Thanks for cleaning it up Steven. As I said it was only a sample script. You can use the idea to make it better. For ex., he would need to do "rm /tmp/tmp$$" at the end of the script and look for the existence of this file before it checks for wc -l etc.,

-Sri

[Bill Hassell]

The sorted du command is going to be your most useful report. Typically there are only one or two directories that grow in problem filesystems so a few days' worth of du listings (I would make a separate listing of each mountpoint) should pinpoint the problem directories. Then look inside and see which files grow the fastest. Use this command to sort the files by size:

ll | sort -rnk5 | more

As far as HP-UX goes, 2 directories will always need cleanup help: /tmp and /var. /tmp is unfortunately misused as a general purpose temp directory (it is supposed to be opsystem use only) so it needs monitoring and regular file and directory removal.

/var is far more complicated since there are many different subsystems that use /var directories. For instance, if you use email or do a lot of printing, /var/mail and /var/spool may grow rapidly. /var/adm always grows because it is full of logfiles. And /var/adm/sw contains rollback files to be used to backup patches.

As with all scripted solutions, automatic removal or truncation is very dangerous without careful inspection of the file and it's purpose.

[Joerg Hinz]

All the solutions you posted above refer to filesystems... scripts which use df or du or whatever and write a mail if something seems to reach a limit are simple.

If you want to monitor files that GROW you'll have to create lists and compare those.

To get a list of large files on your system use find:

# find / -size +1000k -exec du -sk {} \;

That shows all files larger than 1000kb.

Redirect the output into a file.

Then wait some days.

Call again find and redirect the output to file (but with different filename ;)).

Then grab your favourite editor and write a small perl script:
Read the first list into a hash-array (key=filename, value=size).

Now read the new file.

Check for the filename in the hash. If the file is there, compare the size and you see what files grew or shrink. If the file is not in the hash you've got a new file which raised above 1mb.

If you enhance the routine a little bit you can also find large files which were removed.

That's the best solution to your question.

Regards
Joerg