Operating System - HP-UX
1748198 Members
2661 Online
108759 Solutions
New Discussion юеВ

Re: Strange prob with /var filesystem!!!!

 
Sandip Ghosh
Honored Contributor

Strange prob with /var filesystem!!!!

OS - HP-UX 10.20
System - K460
/var - 262 MB (80% full)
EMS - configured
Predictive - configured

Everyday at a particular time the var file system is getting filled up by 100%. And immidiately after that it is getting cleared up automatically. I had tried to catch hold of the process, but I couldn't do it. I have checked the cron logs also, but nothing is there. Can anybody please help me to idetify the process or the file which is getting filled up?

Thanks in advance,

Sandip
Good Luck!!!
15 REPLIES 15
Helen French
Honored Contributor

Re: Strange prob with /var filesystem!!!!

I would think of the following possibilities:
1) lp spooler files in the queue. A large print request which waits in the queue and remove the entry when it's printed.
2) crontab file from a particular user (/var/spool/cron). Check all users cron files and check if any programs causing this problem.
3) Apply latest patches to the system.
4) log files from stm, EMS etc
Life is a promise, fulfill it!
John Poff
Honored Contributor

Re: Strange prob with /var filesystem!!!!

Hi,

Another reason to consider having separate filesystems for /var/tmp, and maybe /var/spool/lp if your print jobs are big enough.

JP
Shannon Petry
Honored Contributor

Re: Strange prob with /var filesystem!!!!

The var directory is used for lots of stuff. Namely:
system administration and
spooling

look at
/var/adm/
check the *acct files and syslog dir for build up.
/var/spool
large print jobs spooled up will usually cause these problems
/var/mail
check for junkmail sent to local server etc....


Run a cronjob every 30 minutes to snapshot the var directory and put the file into a /tmp file.
I.E.
#!/bin/sh
LOG=/tmp/varmon.txt
#create log if not there
touch $LOG
cd /var
echo "System Snapshot" >>$LOG
date >>$LOG
echo "___________________________________" >>$LOG
du -sk * >>$LOG
echo "" >>$LOG ; echo "">>$LOG

Didnt test it, but should work.

Regards,
Shannon
Microsoft. When do you want a virus today?
MANOJ SRIVASTAVA
Honored Contributor

Re: Strange prob with /var filesystem!!!!

Hi Sandip


Like /tmp is used by vi etc as a scratch pad , /var/tmp is used by applications as scratch pad specially if u ahve oracle wiht lots of users , I woul recommend that u clean /var/tmp when the system comeup after booting all the time as this is not done autoamaitcally . also a crude way to find out waht is eating the the space can be to do

cd /var

ls -lR | grep log
and check for the size every 5 seconds to know what is filling up

or you can do like this

while true
ls -lR /var | grep log >> /tmp/abc
sleep 10
done

run this for the time there is a probelm and u can monitor what log files get bulky , but I think the problems is wiht /var/tmp


Manoj Srivastava
A. Clay Stephenson
Acclaimed Contributor

Re: Strange prob with /var filesystem!!!!

One other thought: This might be a remsh'ed process croned from another server.
If it ain't broke, I can fix that.
A. Clay Stephenson
Acclaimed Contributor

Re: Strange prob with /var filesystem!!!!

Because filling up the /var filesystem can bring everything to a halt, you should really consider creating separate mountpoints for /var/spool/lp, /var/mail, and /var/tmp. Your system is much more robust like that. What is probably happening is that some process (probably cron or at) is creating a temp file. Moreover, the process may even unlink the file after opening it so that it is invisible to you but still occupies space until the last process that had it open, closes the file. When the filesystem fills up the process dies and the space is returned.
If it ain't broke, I can fix that.
Sandip Ghosh
Honored Contributor

Re: Strange prob with /var filesystem!!!!

Shiju,

1. If it is from lp-spooler it may not happen at the same particular time.
2. I have looked at the crontab log and couldn't find anything. Whatever it is there, it runs every 15 or 20 min. But the problem occurs particularly at 9:40 AM. And I have looked at all the entries for the cron users and nothing has been set particularly for 9:40 AM.
3. Latest patch has to be applied within 1-2 weeks. Looking for downtime.
4.Looked at the STM and EMS log, but couldn't find anything.

Is it possible to dump something by the ups_mond daemon? As far as I could remeber it is happening shortly after installing the new UPS. I am not sure about that.

Sandip
Good Luck!!!
Leif Halvarsson_2
Honored Contributor

Re: Strange prob with /var filesystem!!!!

Hi
Do you know exact at what time this happens. Perhaps you can get an idea if you start a script short before which monitor the /var subdirectorys where you can suspect something is going on. For example:

for dir in tmp spool cron ...
du -s $dir
sleep 5
done >logfile

Kill the script and browse the logfile. Perhaps you can see if one directory suddenly begins to grow.

Leif Halvarsson_2
Honored Contributor

Re: Strange prob with /var filesystem!!!!

Sorry,

Ow course this must be run in an infinity loop

while true
do
for dir in tmp spool cron ...
do
du -s $dir
sleep 5
done
done >logfile