Urgent: "/" root file system is 100 % used

[praveen..]

i didn't check it before,

but when i gave this command, i got this error:

#mailx -s "subject" praveen@company.com
testing

nospcae on root

please suggest what file i can delete,
this is production server

[praveen..]

# bdf | more
Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol3 524288 524288 0 100% /

[rariasn · Accepted solution]

Hi praveen,

See this link,

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=996967

rgs,
ran

[Robert-Jan Goossens_1]

Hi,

# du -kx / | sort -rn | head -20
what is the largest directory?

# find . -xdev -type f -size +1000 -exec ll {} \;

Regards,
Robert-Jan

[rariasn]

First,

cd /
ll core
mv core /xxxx/alternate_fs

ran

[Arunvijai_4]

Hi Praveen,

remove unwanted files from /. Especially, core files if you have any in /.

-Arun

[Darrel Louis]

Praveen,

Use the following script to check where the big files are under /.

Use as follow:
# cd /
# run the dus.sh script.

And then remove or compress the big-files.

Darrel

[praveen..]

#cd /
#ll
drwx------ 6 root sys 96 Aug 18 2005 .VRTSob
-rw------- 1 root sys 100 Jun 9 2005 .Xauthority
-r--r--r-- 1 bin bin 1222 May 26 2005 .profile
-rw------- 1 root sys 1084 May 12 2005 .sh_history
drwx------ 2 root sys 96 Jun 1 2005 .ssh
drwxr-xr-x 6 root sys 8192 May 5 2005 .sw
-rw-rw-rw- 1 root root 1896 Jan 22 18:50 CTSTATUS.FCS
-rw-r--r-- 1 root sys 0 Apr 21 2005 MANPATH
drwxr-xr-x 2 root sys 96 Jun 1 2005 SD_CDROM
lrwxrwxrwx 1 root sys 5 May 26 2005 ahome -> /home
lrwxrwxrwx 1 root sys 5 May 26 2005 bhome -> /home
lr-xr-xr-t 1 root sys 8 Apr 21 2005 bin -> /usr/bin
drwxrwxrwx 2 root sys 96 May 5 2005 cdrom
drwxrwxr-x 13 root root 1024 Aug 11 2005 cims
dr-xr-xr-x 75 bin bin 8192 Mar 23 08:42 dev
lrwxrwxrwx 1 root sys 5 May 26 2005 dhome -> /home
dr-xr-xr-x 32 bin bin 8192 Jun 5 06:00 etc
dr-xr-xr-x 19 bin bin 8192 Dec 22 07:42 home
lr-xr-xr-t 1 root sys 8 Apr 21 2005 lib -> /usr/lib
drwxr-xr-x 2 root root 96 Sep 8 2004 lost+found
dr-xr-xr-x 2 root sys 96 Oct 8 2005 net
dr-xr-xr-x 71 bin bin 8192 Jan 10 14:22 opt
drwxr-xr-x 11 root sys 8192 Jun 9 03:11 root
dr-xr-xr-x 14 bin bin 8192 Jan 10 09:42 sbin
dr-xr-xr-x 8 bin bin 8192 Mar 30 22:31 stand
drwxrwxrwx 13 bin bin 8192 Jun 9 03:20 tmp
dr-xr-xr-x 28 bin bin 8192 Dec 5 2005 usr
dr-xr-xr-x 28 bin bin 8192 Oct 23 2005 var


other directories are my file systems on which my logical volumes are mounted,

should i empty the /tmp directory?
or suggest

[Robert-Jan Goossens_1]

post the output of

# du -kx / | sort -rn | head -20

[Peter Godron]

Praveen,
this may help:
find / -mountstop -size +1000000c -exec ll {} \;

translate to:
find all files greater than 1000000 bytes, which reside on the / mountpoint.

[Victor BERRIDGE]

Hi
To help you we would need a bit more information on the type of box (WS or Server), the output of bdf also
On a server for sure /tmp is a filesystem so this would not help, what is the size of /dev ?
du -sk /dev

All the best
Victor

[Darrel Louis]

Praveen,

Run the script just posted to you and send the output.

Darrel

[praveen..]

# du -kx / | sort rn | head -10
sort: Cannot open rn

[rariasn]

Hi praveend,

verify /root directory

ran

[rariasn]

Hi praveend,

verify /root directory

cd /root

ran

[Robert-Jan Goossens_1]

-rn :-)

# du -kx / | sort -rn | head -20

[Darrel Louis]

Praveen,

It should be sort -rn

Darrel

[Arunvijai_4]

Check this command as well,

# ls -lR | sort +4 -5nr | more

-Arun

[praveen..]

root@server:[/var/adm/syslog]
# ll
total 46608
-rw-r--r-- 1 root root 64347 Jan 22 17:37 OLDsyslog.log
-r--r--r-- 1 root root 10572682 Jun 9 01:03 mail.log
-rw-r--r-- 1 root root 312628 Oct 18 2005 syslog.bkp
-rw-r--r-- 1 root root 9330421 Jun 9 04:03 syslog.log
-rw-r--r-- 1 root root 921890 Oct 19 2005 syslog.log.bkp
-rw-r--r-- 1 root root 358256 Oct 20 2005 syslog.log.bkp2
-rw-r--r-- 1 root root 1972281 Sep 2 2005 syslog.log.bkup


should i delete all file except mail.log & syslog.log.


#cd /root
# ll
total 176
-rw------- 1 root sys 912 Nov 17 2005 .ICEauthority
-rw------- 1 root sys 74 Nov 17 2005 .TTauthority
dr-xr-xr-x 6 root sys 96 Aug 18 2005 .VRTSob
-rw------- 1 root sys 99 Mar 28 10:52 .Xauthority
drwxr-xr-x 11 root sys 8192 Nov 17 2005 .dt
-rwxr-xr-x 1 root sys 5451 Nov 15 2005 .dtprofile
dr-xr-xr-x 3 root sys 96 Aug 18 2005 .java
-rwx------ 1 root sys 77 Jul 19 2005 .netrc
-r--r--r-- 1 root sys 1237 Jul 19 2005 .profile
drwx------ 2 root sys 96 Jun 1 2005 .ssh
drwxr-xr-x 6 root sys 8192 Jun 29 2005 .sw
drwxr-xr-x 2 root sys 96 Jan 4 16:41 0x410000
drwxr-xr-x 2 root sys 96 Jan 4 16:41 64
-rw-r--r-- 1 root sys 158 Aug 18 2005 CTSTATUS.FCS
drwxr-xr-x 2 root sys 96 Jan 4 16:41 c
-rw------- 1 root sys 0 Jun 9 03:19 dead.letter
-rw-r--r-- 1 root sys 0 Sep 28 2005 dipper.cpm
-rw-r--r-- 1 root sys 389 Sep 27 2005 dipper.out
lrwxr-xr-x 1 root sys 8 Oct 19 2005 ecc -> /usr/ecc
-rw-r--r-- 1 root sys 0 Aug 4 2005 it
-rw-r--r-- 1 root sys 0 Mar 23 08:42 mapfile
drwxr-xr-x 3 root sys 96 Sep 30 2005 rich
-rw-r--r-- 1 root sys 746 Nov 17 2005 sel

[praveen..]

# du -kx / | sort -rn | head -20
505840 /
346168 /LA/db
346168 /LA
346112 /LA/db/ORLAODSD
122152 /sbin
36048 /etc
28224 /etc/opt
22688 /sbin/fs
17144 /etc/opt/resmon
16248 /sbin/fs/vxfs
11568 /etc/opt/resmon/lbin
8624 /sbin/emc
4664 /sbin/fs/hfs
4560 /etc/opt/wbem
4544 /etc/opt/wbem/mof
3928 /etc/opt/samba
3760 /etc/opt/samba/codepages
3336 /etc/opt/resmon/lib
1800 /etc/opt/resmon/log
1592 /etc/opt/cifsclient

[Enrico P.]

find / -type f -size +5000 -xdev -exec ll {} \;|sort -rnk 5

Will show you the file only in your / filesystem order by dimension.

Enrico

[Enrico P.]

If you think you can remove a file, check if the file is in use by any process with the command

fuser file_path

if there are some process using the file then check if you can kill them and after killed these process you can remove the file.

Enrico

[Robert-Jan Goossens_1]

Praveen,

346112 /LA/db/ORLAODSD

# cd /LA/db/

what is ORLAODSD ?

[praveen..]

ORLAODSD is database name.
and its each directory is mounted indiviusaly.

/dev/vg48/la_db_orlaodsd_data_dat01
25624576 25223502 376013 99% /LA/db/ORLAODSD/data/dat01
/dev/vg48/la_db_orlaodsd_data_idx01
9437184 3090797 5949745 34% /LA/db/ORLAODSD/data/idx01
/dev/vg48/la_db_orlaodsd_data_rbs01
5242880 2065769 2978549 41% /LA/db/ORLAODSD/data/rbs01
/dev/vg48/la_db_orlaodsd_data_redoA
786432 737080 46274 94% /LA/db/ORLAODSD/data/redoA
/dev/vg48/la_db_orlaodsd_data_redoB
786432 739664 43851 94% /LA/db/ORLAODSD/data/redoB
/dev/vg48/la_db_orlaodsd_data_tmp01
10485760 6163054 4052544 60% /LA/db/ORLAODSD/data/tmp01

[Robert-Jan Goossens_1]

There is something inside the /LA/db/ directory causing your root filesystem to be 100% full, could be its underneath your filesystem /LA/db/ORLAODSD/data/***

Robert-Jan