Find top 5 space consuming directories.

[Alok_Behria]

Hi All,

SO :- HPUX 11.11

I am trying to find top 5 space consuming directories. In order to achieve this, I have tried the following command, which in turn produced following result. Please suggest, if I am doing anything wrong here..

ptmsap> du -sk * du: cannot open < buyrbtch/env > 818 buyrbtch 4 cconbtch 18 cpmsbtch 49239 dbtbtch 4 ftplbtch 23 intgbtch 69106 invbtch 0 lost+found du: cannot open < pcstbtch/joblog > du: cannot open < pcstbtch/sqllog > du: cannot open < pcstbtch/badlog > du: cannot open < pcstbtch/ctllog > du: cannot open < pcstbtch/ftplog > du: cannot open < pcstbtch/errorlog > 115122 pcstbtch 3625 physbtch 170272 pkgbtch 23 prebtch 211 psabtch 4 rdsbtch 4 sectbtch 4 sectqa du: cannot open < shglbtch/joblog > du: cannot open < shglbtch/sqllog > du: cannot open < shglbtch/ctllog > du: cannot open < shglbtch/badlog > du: cannot open < shglbtch/ftplog > du: cannot open < shglbtch/errorlog > +++ If i do following , it did not return anything ptmsap> du -sk * | sort -nr | head -5 du: cannot open < buyrbtch/env > du: cannot open < pcstbtch/joblog > du: cannot open < pcstbtch/sqllog > du: cannot open < pcstbtch/badlog > du: cannot open < pcstbtch/ctllog > du: cannot open < pcstbtch/ftplog > du: cannot open < pcstbtch/errorlog > du: cannot open < shglbtch/joblog > du: cannot open < shglbtch/sqllog > du: cannot open < shglbtch/ctllog > du: cannot open < shglbtch/badlog > du: cannot open < shglbtch/ftplog > du: cannot open < shglbtch/errorlog > 

[Alok_Behria]

oopes, there you go

du: cannot open < buyrbtch/env >
818     buyrbtch
4       cconbtch
18      cpmsbtch
49239   dbtbtch
4       ftplbtch
23      intgbtch
69106   invbtch
0       lost+found
du: cannot open < pcstbtch/joblog >
du: cannot open < pcstbtch/sqllog >
du: cannot open < pcstbtch/badlog >
du: cannot open < pcstbtch/ctllog >
du: cannot open < pcstbtch/ftplog >
du: cannot open < pcstbtch/errorlog >
115122  pcstbtch
3625    physbtch
170272  pkgbtch
23      prebtch
211     psabtch
4       rdsbtch
4       sectbtch
4       sectqa
du: cannot open < shglbtch/joblog >
du: cannot open < shglbtch/sqllog >
du: cannot open < shglbtch/ctllog >
du: cannot open < shglbtch/badlog >
du: cannot open < shglbtch/ftplog >
du: cannot open < shglbtch/errorlog >
71498   shglbtch
8       shpcbtch
4       snetbtch
109129  srpbtch
1253    suppbtch
62      tmsbtch
4       unvrbtch
tbx4 oraptsap /db/ptmsap/batch
ptmsap> du -sk * | sort -nr | head -5
du: cannot open < buyrbtch/env >
du: cannot open < pcstbtch/joblog >
du: cannot open < pcstbtch/sqllog >
du: cannot open < pcstbtch/badlog >
du: cannot open < pcstbtch/ctllog >
du: cannot open < pcstbtch/ftplog >
du: cannot open < pcstbtch/errorlog >
du: cannot open < shglbtch/joblog >
du: cannot open < shglbtch/sqllog >
du: cannot open < shglbtch/ctllog >
du: cannot open < shglbtch/badlog >
du: cannot open < shglbtch/ftplog >
du: cannot open < shglbtch/errorlog >

[Dennis Handly]

(Please go back and delete the scrambled output.)

>du: cannot open < buyrbtch/env >

You need to be root or have the right permissions.

>du -sk * | sort -nr | head -5

You then need to sort the first numeric key and select the top 5 and dump the errors.  And -x to not cross mount points.

du -kxs * 2> /dev/null | sort -nr -k1,1 | head -5

[Alok_Behria]

You seems to be on target, evertime I ask the question. I observerd , one strage thing, when I fired the suggested command, it took almost 25 seconds to produced results. I would also like to know, the purpose of "-kxs " attribute. If I am right, the command ( "sort -nr -k1,1)is extract the first number and sort them, right? I also wonderd, why we need to dump errors ? can't we have a command, which work against directories, also, is there any command, which we can use to measure the elapsed time of command run. Thanks once again for your help.

[James R. Ferguson]

---

@Alok_Behria wrote:

I observerd , one strage thing, when I fired the suggested command, it took almost 25 seconds to produced results. I would also like to know, the purpose of "-kxs " attribute. If I am right, the command ( "sort -nr -k1,1)is extract the first number and sort them, right? I also wonderd, is there any command, which we can use to measure the elapsed time of command run. Thanks once again for your help.

---

The "delay" is the process of recursively walking through the directory you specified and calling 'stat()' for each file and directory to obtain the number of blocks allocated to it.

If you look at the manpages for 'du(1)' you will see that '-kxs' says to report in 1024 blocks (k); to (s)ummarize a grand total for the directory specified; and *not* (-x) cross mountpoints (i.e. follow symbolic links from one mountpoint to another).

The sort says to use field (column) one (1) of the input from the beginning of field-one until its end as the sort's key ('k').  The field is designated as numeric ('n') for sorting comparisons.  The output should be ordered in decending order ('-r' for "reverse) from the largest value to the smallest value.  Since you didn't specify a field delimiter (with '-t'), whitespace is assumed.

Once again, here, *read* the manpages for the various commands to learn their details!

As for timing a command or script, you can use 'time' or 'timex' like this:

timex du -xks /mydi

Regards!

...JRF...

[Bill Hassell]

It's also important to know that your results are inaccuarate because you do not have permission to read the directories. Thus, many directories are missing from the totals. It doesn't make any sense to look at a few directories and draw any conclusions about disk space. Run the script as root and you will see the top 5 directories. This technique is exactly how you manage growing directories sych as /var. The delay before you see the results is due to the hundred (or thousands) of directories and files that must be queried.

[Dennis Handly]

>I observed, one strange thing, when I fired the suggested command, it took almost 25 seconds to produced results.

You're lucky it didn't take 10s of minutes.  ;-)

>the purpose of "-kxs " attribute.

The -x was to stop it from crossing filesystems, if you have a mount point in some subdirectory.

>the command ( "sort -nr -k1,1)is extract the first number and sort them, right? I also wondered, why we need to dump errors?

Right.  So they don't clutter the screen.  Also, since they go to stderr, they aren't passed to sort nor head.

>can't we have a command, which work against directories

Does what with those directories?  That's what du -s does?