File System

[Amitav]

My /var FS gets filled by almost 96% everday.
How can i reduce this, except increasing the FS...??

[MikeL_4]

Have you checked your log files:
/var/log and /var/adm/syslog ?

Is /var/adm/wtmp getting to big ?

[James R. Ferguson]

Hi Amitav:

Begin by finding out what process or processes is responsible. What files in what subdirectories do you find?

You can look for the largest directories with:

# du -k /var|sort -knr1,1|more

Look for recent files (those created in the last 24-hours) with:

# find /var -xdev -type f -mtime 0 -exec ls -l {} \;

Regards!

...JRF...

[lawrenzo]

yes as mentioned ...

check the size of wtmp and sulog ...

if the wtmp file is growing quickly then identify what is login into your system and also investigate a rapid growing sulog file.

run

find /var -xdev |xargs ls -ld |sort -rnk5

to find the largest file to the smallest on the filesystem.

investigate what procs are writing to the fs by running:

ps -p "$(fuser -c /var 2>/dev/null)"

let us know what the top offending files are ...

[Roberto Arias]

Hi all

In my system I have a Filesystem for /var/adm/crash. You can remove files of /var/tmp and empty logfiles of /var/opt//....

check size of /var/adm/syslog/mail.log,syslog.log, OLDsyslog and /var/adm/btmp,wtmp nettl00.log

i hope help you with tis

[Amitav]

PID TTY TIME COMMAND
873 ? 3:17 sshd
688 ? 0:00 ptydaemon
702 ? 0:00 hotplugd
725 ? 0:00 ntl_reader
726 ? 9:12 netfmt
1194 ? 39:50 snmpdm
1268 ? 43:49 cimserver


These are the processes which are being at present...

[Amitav]

PID TTY TIME COMMAND
873 ? 3:17 sshd
688 ? 0:00 ptydaemon
702 ? 0:00 hotplugd
725 ? 0:00 ntl_reader
726 ? 9:12 netfmt
1194 ? 39:50 snmpdm
1268 ? 43:49 cimserver


These are the processes which are being written by FS at present...

[Mridul Shrivastava]

Please run the following command to find out the big files,

cd /var
pwd
find . -type f -xdev -size +10000000c -exec ll {} \;

If it shows lots of file then increase on Zero in the command syntax.

once you locate those files which are big .. you may decide to nullify them or move them based on the file types.

If they are log files then they can be nullyfied, and if there is a crashdump under /var/adm/crash then take a tape backup of the same and remove it from the server.

If doubts then post the largest files present in the /var.

[A. Clay Stephenson]

The list of processes is rather useless. What you need to determine are the files that are growing rapidly in size. Are ary of the log files growing rapidly? If so, tail the file and see if an error or information message is being constantly repeated. That will lead you to the problem.

Another common problem is simply bad code. It is very common (and normal) to use /var/tmp for temporary files. You may have a situation in which processes are dying and not cleaning up their files. Using kill -9 will have the same effect because a SIGKILL cannot be ignored and no cleanup is done.

[Amitav]

Hi Stephenson,

The cron log is growing, and as suggested by u, i viewed the file, but no error was found...

[A. Clay Stephenson]

Ok. Does the growth of the cron log alone explain why the file system is filling up so fast? It is normal for the cron log (as well as many others) to grow but cron jobs are spawned at most once per minute so even if you had 40 jobs spawned every minute (an extreme case) the cron log growth should not be all that rapid.

What you are seeing may also be normal but you do not have automated processes in place to trim logs.

For the moment, I would look for other rapidly growing files.

[Amitav]

I doubt that ther are any automated process to trim the log....the other files which are growing are wtpm & btmp...

[A. Clay Stephenson · Accepted solution]

Ok, now we are getting somewhere. wtmp and btmp indicate new connection attempts and if btmp is growing that means that you are receiving failed (bad passwords) connection attempts. You have a remote host (or several of them) that are trying to connect to your box. One way to help track that down is to enable inetd logging (note that the log will grow very rapidly) but that log will contain the IP addresses of the remote hosts that are trying to connect.

[A. Clay Stephenson]

By the way, it is very good practice to divide /var into multiple filesystems such as /var/mail, /var/tmp, /var/spool, /var/adm/sw so that the /var filesystem itself never fills up. For example, filling up /var/spool/lp will stop printjobs but will not bring the entire system down as it would if /var/spool/lp were simply part of the /var filesystem.