topic Re: script for total files size in Operating System - HP-UX

script for total files size

himacs — Sun, 04 Oct 2009 10:30:12 GMT

Hi Admins,

I want to get the total size of files dated Oct 4 under /data.Find the script which i tried.

#!/usr/bin/sh
total=0
files=du -sk `ls -lrt /data | grep "Oct 4"|awk '{print $9}'`
for filesize in $files
do
total=$total+$files
done

But m ended up with error.

./test[3]: -sk: not found.

Plz suggest on this.

regards
himacs

Re: script for total files size

Duncan Edmonstone — Sun, 04 Oct 2009 10:51:52 GMT

This shows a fairly major lack of understanding of shell scripting - I suggest you spend some time looking at some online training material on doing shell scripting:

http://docs.hp.com/en/B2355-90046/index.html

why are you using du ?

du will show you disk usage. *not* file size - these are not the same thing...

if you really want to gather disk sizes in this rather crude way, I guess:

#!/usr/bin/sh
total=0
sizes=`ls -lrt /data | grep "Oct 4"|awk '{print $5}'`
for filesize in $sizes
do
(( total = $total + $filesize ))
done
echo $total

might work for you (untested)

HTH

Duncan

Re: script for total files size

Duncan Edmonstone — Sun, 04 Oct 2009 10:55:07 GMT

now I'm confusing myself:

> if you really want to gather disk sizes

should of course read:

if you really want to gather and total file sizes

HTH

Duncan

Re: script for total files size

Matti_Kurkela — Sun, 04 Oct 2009 10:58:06 GMT

> files=du -sk `ls -lrt /data | grep "Oct 4"|awk '{print $9}'`

The above line is equivalent to:

files=du
-sk `ls -lrt /data | grep "Oct 4"|awk '{print $9}'`

The $files variable got assigned the string "du" and then the system attempted to execute a command named -sk, which of course did not exist.

You would need to use multiple sets of backquotes, which gets confusing and is prone to errors. Fortunately, there is another way: you can use $( ... ) instead of `...`. That is easier to cascade:

files=$(du -sk $(ls -lrt /data | grep "Oct 4"|awk '{print $9}'))

Your script still has some limitations:

1.) If there are more files created on Oct 4 than can fit on a single command line (run "getconf ARG_MAX" to see the maximum command line length in bytes), your script will fail. But if you already know that the amount of files is relatively small, that is not important.

2.) If there are sub-directories under /data which have been changed on Oct 4, this may skew your results. If there are no subdirectories, not a problem.

3.) The script calculates files dated Oct 4, but Oct 4 _of which year_? If you're sure all files are less than 1 year old, this should not be a problem.

MK

Re: script for total files size

Dennis Handly — Sun, 04 Oct 2009 10:59:13 GMT

You need to stop using archaic `` and use $() so you can nest them:
files=$(du -sk $(ls -lrt /data | grep "Oct *4" | awk '{print $9}'))

I'm not sure why you are using du(1) then totaling. Are you worried about sparse files?

total=$(ll /data | awk '
BEGIN { total = 0 }
{ total += $5 }
END { printf "%1.0f\n", total }')

Re: script for total files size

Dennis Handly — Sun, 04 Oct 2009 11:07:49 GMT

>MK: The above line is equivalent to:
files=du
-sk `ls -lrt /data | grep "Oct 4"|awk '{print $9}'`

Almost, they are on the same line.

>You would need to use multiple sets of backquotes, which gets confusing and is prone to errors.

I'm not aware of any way to nest ``, so perhaps the correct statement is "impossible"?
(You could split them in multiple steps.)

Oops, I lost that "Oct 4" step:
total=$(ll /data | awk /Oct *4/ '
BEGIN { total = 0 }
{ total += $5 }
END { printf "%1.0f\n", total }')

Re: script for total files size

himacs — Sun, 04 Oct 2009 11:54:28 GMT

Hi Admins,

Thanx for the replies.Since m new into scripting world, i have limited knowledge on that.
So Denis, before testing what u suggested i need to refer that commands.

And Duccan ,Thanx for the document.
And below script i modified.But below error.

#!/usr/bin/sh
files=`ls -lrt /data | grep "Oct 4"|awk '{print $9}'`
size=`du -sk $files`
total=0
for i in $size
do
total=$total+$size
done
echo $total

du: FCT_LOANS_CONTRACTS_20091001.DAT.gz: No such file or directory
du: FCT_ICCF_DETAILS_20091001.DAT.gz: No such file or directory
du: FCT_INSTRUMENT_TXN_20091001.DAT.gz: No such file or directory
du: FCT_DISBURSEMENT_20091001.DAT.gz: No such file or directory
du: DIM_PDC_20091001.DAT.gz: No such file or directory
du: DIM_PERIOD_CODE_STATUS_20091001.DAT.gz: No such file or directory
du: FCT_ALL_AC_ENTRIES_20091001.DAT.gz: No such file or directory
du: DIM_MIS_DATES_20091001.DAT.gz: No such file or directory
du: FCT_LOAN_CONTRACT_LINKAGES_20091001.DAT.gz: No such file or directory
du: FCT_LIMIT_UTIL_20091001.DAT.gz: No such file or directory
du: DIM_MIS_20091001.DAT.gz: No such file or directory
du: FCT_LIMITS_20091001.DAT.gz: No such file or directory
du: DIM_MARKET_20091001.DAT.gz: No such file or directory
du: FCT_LC_INFO_PARTIES_20091001.DAT.gz: No such file or directory
du: FCT_LCS_CONTRACTS_20091001.DAT.gz: No such file or directory

I know you suggested not to use du -sk command.But when i tested the command in shell prompt i got desired output.

#du -sk `ls -lrt|grep "Oct 2"|awk '{print $9}'`
353 FCT_VEHICLE_DETAILS_20091001.DAT.gz
109 FCT_CONTRACT_STATUS_20091001.DAT.gz
4090 FCT_CONTRACT_MIS_20091001.DAT.gz
572 FCT_GL_ACCBREAKUP_20091001.DAT.gz
5243 DIM_CUSTOMER_20091001.DAT.gz
14426 FCT_ADV_SCHEDULE_20091001.DAT.gz
798 FCT_LOANS_CONTRACTS_20091001.DAT.gz
702 FCT_ICCF_DETAILS_20091001.DAT.gz
570 FCT_INSTRUMENT_TXN_20091001.DAT.gz
38 FCT_DISBURSEMENT_20091001.DAT.gz

Plz suggest on this

regards
himacs

Re: script for total files size

Dennis Handly — Sun, 04 Oct 2009 12:20:00 GMT

>files=`ls -lrt /data | grep "Oct 4" | awk '{print $9}'`
size=`du -sk $files`

This fails because you are looking at /data but the awk is only getting the filenames, so the du(1) fails.

You'll need to cd to /data, or change to use:
size_files=$(du -sk $(ls -lrt /data | grep "Oct *4" | awk '{print "/data/" $9}'))

>when I tested the command in shell prompt I got desired output.

Because you were in /data/.

>353 FCT_VEHICLE_DETAILS_20091001.DAT.gz

And you will have to extract field 1 from this to do the summation.

Re: script for total files size

himacs — Sun, 04 Oct 2009 14:15:21 GMT

Hi admins,

Thanx for ur time.

Below is the script i written which given correct output.

#!/usr/bin/sh
files=`ls -lrt /data | grep "Oct 2"|awk '{print "/data/" $9}'`
total=0
for i in $files
do
var1=`du -sk $i|awk '{print $1}'`
#echo $var1
total=`expr $total + $var1`
done
echo $total

It will work ,if no sub folders exists

Please tell me how to modify the same if subfolder exists

regards
himacs

Re: script for total files size

James R. Ferguson — Sun, 04 Oct 2009 14:28:53 GMT

Hi Himacs:

> Please tell me how to modify the same if subfolder exists

To accommodate this it is easiest to use 'find -type f' to recursively descend subordinate directories below the starting path.

One way to limit what you have to the date of October 4 is to do:

# touch -amt 200910040000 /tmp/ref1
# touch -amt 200910042359 /tmp/ref2

# find /path -type f $ -newer /tmp/ref1 -a ! -newer /tmp/ref2 $

This would find only files that have been modified during the 24-hour period of October 4th. You should read the manpages for 'touch' and for 'find' for a better understanding.

If you don't want to use 'find()' you could use 'ls' recursively (with its '-R' option). In that case, though, you will have to filter out only files by examining the first character of a long output. Hint: directories begin with "d" and regular files with "-".

Regards!

...JRF...

Re: script for total files size

Raj D. — Sun, 04 Oct 2009 18:06:55 GMT

himacs,

> I want to get the total size of files dated Oct 4 under /data.

Try this:
# cd /data ; ls -l | grep "Oct 4" | awk 'BEGIN {sum=0} {sum+=$5} END {print sum}'

Cheers,
Raj.