Displaying output from df -k in GB instead of KB

[Patrick Ware_1]

Hello all,

Code below:

[CODE]

echo "Oracle Filesystems"
echo "------------------"
echo
for j in `df -l -k |grep total|grep ora|grep -v storage|grep -v vg00|awk '{print $1}'`
do
echo $j is `df -l -k $j |grep total|grep ora|grep -v storage|grep -v vg00|awk -F":" '{print $2}'|awk '{print $1}'` KB
done
echo

[/CODE]

Here is the output:

Oracle Filesystems
------------------

/ora_1 is 369437312 KB
/ora_2 is 369438640 KB
/ora_3 is 366768144 KB


I want to dsiplay the output of df -k in GB instead of KB. How would I do that?

[James R. Ferguson · Accepted solution]

Hi Patrick:

Bill Hassell has a 'bdf' script which allows output in MB or GB and more importantly handles the case of multi-line output for any filesystem.

Why re-invent the wheel, when Bill offers his documented script here:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1124262

You can also select only the filesystems you want reported when you run it, just like the standard 'bdf'.

Regards!

...JRF...

[Pete Randall]

Bill Hassell has enhanced his "bdfmegs" script to display GB output. See:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1048509

I would think you could use that.


Pete

[Tim Nelson]

James is too quick. I would also reference the Bill H. script. Works great, forum approved :)

[Pete Randall]

And here's the latest version of "bdfmegs":

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1124262


Pete

[Hein van den Heuvel]

Patrick, you have the 'right solution' already.
Good.

But indulge me, and take an other look at the original script.
It runs 6 processes on the first real line,
and then N times 6 more for each oracle mount point, for a grand total of 24 in the example, all this for data which is right there for the grab / filter all along.

Stuff like that makes performance folks cry, cringe, or snicker depending on their mood.

Check out this 'one liner'.
It does the same job with 2 commands:

df -k -l | awk 'BEGIN{print "Oracle Filesystems\n------------------\n"} END{print ""} /ora/&&!/vg00/{print $1,"is",$5,"Kb"}'

Now change the main print to printf and pass $5/1024 instead of $5 as found and voila!

In script form...

------------------
#!/bin/sh
df -k -l | awk '
BEGIN {print "Oracle Filesystems\n------------------\n"}
END {print ""}
/ora/ && !/vg00/ && !/storage/ {
printf ("%-20s is %5.1f Gb\n",$1,$5/1024)
} '
------------------

Of course it matters little for a task like posted, but it's the principle of things.

Also, I'd like to think that if the task had been solved along the lines above, then any Unix hacker in your environment, and many a Windoze person, could have handled the improvement request.

Hope this helps someone some day,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Bill Hassell]

For your example:

> /ora_1 is 369437312 KB
> /ora_2 is 369438640 KB
> /ora_3 is 366768144 KB

You can get not only the selected filesystems but also a sum total for them from bdfmegs:

echo "Oracle Filesystems"
echo "------------------"
bdfmegs -g -s /ora_*

[Patrick Ware_1]

All who have shown me bdfmegs, thank you! I like the way it works.

Hein van den Heuvel,

I like your method. Thank you too!

One thing I noticed is that I had to make the following line:

printf ("%-20s is %5.1f Gb\n",$1,$5/1024)

Look like this:
printf ("%-20s is %5.1f Gb\n",$1,$5/1024/1024)

To get true GB readings.

[Patrick Ware_1]

Also, you seem to be on a roll. How would one make this one more efficient:

df -l -k |grep total|grep -v ora|grep -v ebr|grep -v vg00|awk -F":" '{print $2}'|awk '{print $1}' > sizes.flatfiles

[Patrick Ware_1]

I figured it out:

df -k -l | awk '
/total/ && !/ora/ && !/vg00/ && !/ebr/ {
printf ("%5f\n",$5) >> "sizes.flatfiles"
} '


Too bad i can't give myself points!

[Hein van den Heuvel]

Good catch, and good fix.
My script reported Mb, not Gb.

>> I figured it out:

Excellent.

>> Too bad i can't give myself points!

Just find grab a query I created and say hello! Nah...

Seriously, I am very pleased you considered the alternative and checked out how it worked. Good work!
I had wondered whether it would be a total waste of time to add my reply, but couldn't help myself. No I'm redeemed :-).

As reward I'll show how to do grand total. :-).
I added a variable t (total) which accumulated all KB values in the main code.
Then at the end, where previously is printed just an empty line, it now prints that total.. divived by 1M.


#!/bin/sh
bdf -k -l | awk '
BEGIN {print "Oracle Filesystems\n------------------\n"}
END {printf ("\nTotal %3.1fGb\n\n", t/1024/1024)}
/ora/ && !/vg00/ && !/storage/ {
t+=$5;
printf ("%-20s is %5.1f Gb\n",$1,$5/1024/1024)
} '


Cheers,
Hein.

[Dennis Handly]

>How would one make this one more efficient:
df -l -k |grep total|grep -v ora|grep -v ebr|grep -v vg00|awk

If you want to do a part way job, you can combine the grep -v:
df -l -k |grep total|grep -v -e ora -e ebr -e vg00 | awk

You may have to be careful if your strings are found as substrings. You can add -w to fix that.

[Patrick Ware_1]

Looks like I ran into a little issue:

# df -l -k /ora05 | grep total
/ora05 (/dev/vg05/lvol1 ) : 368868648 total allocated Kb

# df -l -k /ora06 | grep total
/ora06 (/dev/vg06/lvol1) : 366767600 total allocated Kb

If you look at the middle part where the logical volume is listed on each, you can see a difference that will throw awk off. In the first listing, there is a space between the lvol name , and the closing parenthesis ")". In the second listing, there is no space. This means that the $5 is the word "total" instead of the numerical size. As a result of this, I see that the calculations are off like below due to this extra space:


/ora05 is 360223.289 Mb
/ora06 is 0.000 Mb

Do you know how to account for this?

[Patrick Ware_1]

I guess what I'm trying to figure out is how can I do an if, then, else type of logic to this?

[Dennis Handly]

>I guess what I'm trying to figure out is how can I do an if, then, else type of logic to this?

You should be able to do $(NF-3) to get that field.
$ df -l -k | grep total | awk '{print $(NF-3)}'

[James R. Ferguson]

Hi Patrick:

Seeing 'grep' in a pipeline to 'awk' says you are WASTING resources. 'awk' is a pattern-matching engine!

>Dennis: You should be able to do $(NF-3) to get that field.
$ df -l -k | grep total | awk '{print $(NF-3)}'

Eliminate the extra process:

# df -l -k | awk '/total/ {print $(NF-3)}'

...You even save your fingers from typing :-)

Regards!

...JRF...

[Patrick Ware_1]

Thank you two! I am now working on incorporating what you have showed me into what Hein van den Heuvel has provided. If anyone wants to save me some time, I wouldn't mind.

[Patrick Ware_1]

Ok, I figured it out:

df -k -l | awk '
/total/ && !/ora/ && !/vg00/ && !/ebr/ {
printf ("%5f\n",$(NF-3)) >> "sizes.flatfiles"
} '

[Patrick Ware_1]

Made another variation of Hein van den Heuvel's grand total script:

df -k -l | awk '
BEGIN {print "\nFlat Files\n------------------" >> "sizes.report"}
END {printf ("\n------------------\nTotal %3.3f Tb\n\n", t/1024/1024/1024) >> "sizes.report"}
/total/ && !/ora/ && !/vg00/ && !/ebr/ {
t+=$(NF-3);
printf ("%-35s is %5.3f Gb\n",$1,$(NF-3)/1024/1024) >> "sizes.report"
} '

df -k -l | awk '
BEGIN {print "\nOracle Filesystems\n------------------" >> "sizes.report"}
END {printf ("\n------------------\nTotal %3.3f Tb\n\n", t/1024/1024/1024) >> "sizes.report"}
/total/ && /ora/ && !/vg00/ && !/storage/ {
t+=$(NF-3);
printf ("%-35s is %5.3f Gb\n",$1,$(NF-3)/1024/1024) >> "sizes.report"
} '

df -k -l | awk '
BEGIN {print "\nEBR Filesystems\n------------------" >> "sizes.report"}
END {printf ("\n------------------\nTotal %3.3f Tb\n\n", t/1024/1024/1024) >> "sizes.report"}
/total/ && /ebr/ && !/vg00/ {
t+=$(NF-3);
printf ("%-35s is %5.3f Gb\n",$1,$(NF-3)/1024/1024) >> "sizes.report"
} '

The code above creates sizes.report

sizes.report output:
---------------------------------------------
Flat Files
------------------
/usr/local is 2.291 Gb
/usr/local2 is 0.118 Gb
/var/mqm is 9.170 Gb

------------------
Total 0.011 Tb

Oracle Filesystems
------------------
/usr/local/oracle/10.2.0 is 7.795 Gb
/usr/local/oracle/9.2.0 is 5.686 Gb
/usr/local/oracle is 5.708 Gb
/usr/local/oracle2 is 1.907 Gb

------------------
Total 0.020 Tb

EBR Filesystems
------------------
/opt/app/ebr1 is 2029.520 Gb
/opt/app/ebr2 is 2029.520 Gb
/opt/app/ebr3 is 2029.520 Gb
/opt/app/ebr4 is 2031.505 Gb
/opt/app/ebr5 is 2029.520 Gb
/opt/app/ebr6 is 2029.520 Gb

------------------
Total 11.894 Tb

---------------------------------------------
end sizes.report output


The code below processes the file, and outputs a total of the totals:

cat sizes.report | awk '
BEGIN {print "\nTotal Of All \n------------------" >> "sizes.report"}
END {printf ("\nTotal %3.3f Tb\n\n", t) >> "sizes.report"}
/Total/ {
t+=$(NF-1);
#printf ("%5.3f Tb\n",$(NF-1) >> "sizes.report")
} '

The result is appended to the end of the file sizes.report:

Total Of All
------------------

Total 11.925 Tb

[Dennis Handly]

>Made another variation of Hein van den Heuvel's grand total script:

Here is why you want to use grep and awk. You have three scripts that are very much alike. While you can use awk -v to pass in your report title (or do it outside), it is harder to get your RE to take a variable.

See my solution in this thread:
http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=1158757
If you want the patterns to be variables:
awk -v p1="^003" -v p2="^abc" -v p3="^a13" -v p4="^next" '
$0 ~ p1 || $0 ~ p2 || $0 ~ p3 {

You can also have functions in awk.

[Patrick Ware_1]

Thanks Dennis for your input. I must ask, what do you mean by "RE"?

[James R. Ferguson]

Hi Patrick:

"RE" is a common abbreviation for Regular Expression. A more descriptive abbreviation is 'regexp'. In fact, the manpges for 'regexp(5)' provide a good introduction.

http://www.docs.hp.com/en/B2355-60105/regexp.5.html

Regular expressions exist in many languages and utilities. 'grep' is the most familiar case to many. The C language, 'awk', 'sed' and particularly Perl offer regular expressions, Perl having one of the most robust engines built naturally into it.

Regards!

...JRF...

[Patrick Ware_1]

Thanks all! I am now reading an AWK tutorial to drive home all the pointers you folks have given me.

[Hein van den Heuvel]

>> I am now reading an AWK tutorial to drive home all the pointers you folks have given me.

Excellent! Did you already get to the chanter on arrays?


Check this out....

df -l -k | awk -f df.awk

------- df.awk --------------

BEGIN {
header["/ora"]="Oracle Filesystems";
header["/ebr"]="EBR Filesystems";
header["none"]="Flat Files";
}

END {
for (name in header) {
t = 0;
print "\n\n" header[name] "\n-------------\n";
for (j=0; j t += size[name,j];
printf ("%-20s is %5.1f Gb\n",mount[name,j],size[name,j]/1024/1024)
}
grand += t;
printf ("\nTotal %5.1f Tb\n",t/1024/1024/1024);
}
printf ("\nGrand Total %5.1f Tb\n",grand/1024/1024/1024);

}

/total/ && !/vg00/ {
kb = $(NF-3);
name = "none";
for (x in header) {
if ($1 ~ x) name = x;
}
j = lines[name]++;
mount[name,j] = $1;
size[name,j] = kb;
}

------------------

Cheers,
Hein.

[Patrick Ware_1]

Not yet, but this may be a good case study. Thanks!