/usr became full

[James Ellis_1]

I was just notified that /usr became full and checking things out under this directory, I don't see anything out of the ordinary. But I see links to /var/adm, /var/tmp, /var/mail, /var/preserve, /var/spool, and /var/tmp.

Should I be concerned with these links, because /var can grow.

What other ways I can check out why /usr is at 100%? My /usr is 3.2G but its full now. Any help?

Thanks.

[Hai Nguyen_1]

Search for any big core file:

# find /usr -type f -name core

Then delete if any found.

Also run:

# du -sk /usr/local
to see how big this directory is. I suspect that the trouble-making file(s) is in here.

Good luck.

Hai

[John Kittel]

the fullness or lack thereof in the deirectories that are linked will not affect the fullness of /usr.

[RAC_1]

The links would not affect the /usr.

/usr is static fs, and should not increase, unless someone is dumping files there.

du -sk *|sort -nr will give you the sorted list of who is how much??

Also as mentioned earlier, you can delete core files if any.

Anil

[James Ellis_1]

Hai,

This is what I got. Executing #du -sk /usr/local produces 258672 /usr/local and this is quite small.

Doing the #find /usr -type f -name core produces nothing.

I already cleaned out 4 core files, but these files did not impact the size of /usr when I did a BDF.

It has to be something else. On another note, doing #last, I noticed a bunch of userid doing ftp, could it be possible that an FTP file got misdirected?

Thanks.

[Naveej.K.A]

hi,

i feel its uncommon for those links to exist in /usr. But even if they are there, its not going to do any harm in filling up /usr. what you could do is probably check for some *tar files or some device file which could have created by mistake in the /usr directory.

regds
Naveej

[John Kittel]

You say you were just notified that /usr is full. If you mean that a user notified you of this, perhaps they can tell you what they were doing or trying to do that alerted them to the situation. I think it is uncommon that a general user would be doing anything that would put files in /usr and yet in order for them to notice or care that it was full perhaps they were trying to and perhaps this is why it is full.

[James Ellis_1]

Here is the result of du -sk * | sort -nr

377224 lib
258672 local
218443 uni
206288 legato
77104 sbin
72504 dt
47440 conf
42888 share
35472 contrib
28456 sam
26440 bin
16488 ccs
8920 include
7496 newconfig
7448 obam
6672 lbin
4776 tsm
1320 old
712 vue
208 examples
24 etc
0 tusc
0 tmp
0 tftpdir
0 spool
0 pub
0 preserve
0 news
0 netls
0 man
0 mail
0 lost+found
0 keysh
0 hpC2400
0 adm

What can I do about /lib and other top directory hogs?

Thanks.

[RAC_1]

Dont'touch those directories, those all are required.

What does bdf /usr says?

Anil

[James Ellis_1]

John,

Actually I receiced a page that /usr was full. I agree, most users don't havbe anything to do with /usr.

When I tried to do a #last | more, I got this message:

ksh: 10245 Memory Fault

What does this mean. The command "last" was working fine about an hour ago.

[James Ellis_1]

bdf /usr says


3194880 3194880 100% used etc

[John Kittel]

just an observation... the sum of the sizes reported in your du -sk... do not seem to add up to anywhere close to 3.2GB.

[RAC_1]

Is user a seperate FS???

Again, run
find /usr -type f -name core
ll -ltr|sort -nk5

Anil

[Michael Schulte zur Sur]

Hi,

have you deleted any open files lately? I would suggest, if possible, to get the machine into single user to exlude hidden, open files.

greetings,

Michael

[Sridhar Bhaskarla]

Hi James,

From your du -sk output, /usr should only be of 1444995 MB used (assuming that none of the directories listed were filesystems).

So, I believe there was a huge active file that was removed but is still opened by some process. I would run 'lsof' at this time and find all the open files on /usr filesystem.

lsof /usr > /tmp/lsof_of_usr.out

I would run the following command to find the culprits.

du -kx /usr |sort -n

-Sri