topic Re: Higheset disk utilizaing users in Operating System - HP-UX

Higheset disk utilizaing users

Spark_2 — Thu, 12 Nov 2009 04:30:26 GMT

Hi,

I have done quite a research on various ITRC threads. With help of those, I can find the disk's utilization using sar -d, iostat and glance -u. In my box, there are two internal disks only which have partitions like /orahome
and /oraclient mounted on it. And the utilization is shooting to as high as 100%.

Is it possible to identify the exact highest users please?

Re: Higheset disk utilizaing users

Aneesh Mohan — Thu, 12 Nov 2009 05:06:12 GMT

Hi,

You need to use glance for this.

Please follow Pete advice on this in the below thread.
http://forums13.itrc.hp.com/service/forums/questionanswer.do?threadId=1012505&admit=109447627+1258002064211+28353475

Regards,
Aneesh

Re: Higheset disk utilizaing users

Michael Steele_2 — Thu, 12 Nov 2009 05:11:01 GMT

Hi

Use sar -d for this and

%busy
Portion of time device was busy servicing a request;

avque
Average number of requests outstanding for the device;

r+w/s
Number of data transfers per second (read and writes) from and to the device;

blks/s
Number of bytes transferred (in 512-byte units) from and to the device;

avwait
Average time (in milliseconds) that transfer requests waited idly on queue for the device;

avserv
Average time (in milliseconds) to service each transfer request (includes seek, rotational latency, and data transfer times) for the device.

When avwait is > than avserv that disk is bottlenecked

Re: Higheset disk utilizaing users

Spark_2 — Thu, 12 Nov 2009 06:11:55 GMT

Thank you for your replies.

I am able to identify the utilization per disk using glance, sar and iostat
Now I want to know who exactly is using. How to find the processes, users who are utilizing the disk highest

Re: Higheset disk utilizaing users

Johnson Punniyalingam — Thu, 12 Nov 2009 06:36:17 GMT

>>How to find the processes, users who are utilizing the disk highest<<

its not possible, as per my knowledge,

but you can narrow down,

using lsof & fuser command to check which process & userid hold file system

fuser -c /

man lsof and man fuser

Re: Higheset disk utilizaing users

Aneesh Mohan — Thu, 12 Nov 2009 06:43:45 GMT

You can sort the i/o utilisation list based on the process.

I dont have a system with glance installed now..please try Pete advice to get the same.

From the GPM Main window, select Reports > Process List. In the Process List window, select Configure > Sort Fields. That brings up a window where you highlight the field you want to sort on and move that field to the leftmost side - the field are sorted from left to right so moving Phys IO Rate all the way to the left will show you your most I/O intensive process.

Aneesh

Re: Higheset disk utilizaing users

Hakki Aydin Ucar — Thu, 12 Nov 2009 06:51:15 GMT

Hi,
Your question remind me of a script I know; spacehogs: Try it:

#!/bin/sh

echo "Enter FS or Path i.e. /tmp "
read patika

find $patika -type f -exec /bin/ls -ls {} ';' | awk '
{ using[$10] += $1 }
END { for (name in using)
{ print using[name], name } }' | sort -nr | head |tee /tmp/trash_go
exit 0

Re: Higheset disk utilizaing users

Michael Steele_2 — Thu, 12 Nov 2009 07:39:57 GMT

Hi

As stated your request can only be narrowed down.

a) Using sar -d and based upon the test avwait > avserv note the disk.
pvdisplay -v disk |more - Note the file systems.

# cd filesystem
# touch 04230000 mfs
# ll
..... 0 Apr 23 00:00 mfs

Example of using the find on filesystem ..
# find / -type f -newer /mfs -exec ll {} \;

This will give you a report of all data files currently opened any updated after

..... 0 Apr 23 00:00

Or you can use lsof and fuser.

Re: Higheset disk utilizaing users

Spark_2 — Thu, 12 Nov 2009 09:04:46 GMT

Hakki,

The script seems to be pulling biggest files.

I was trying to find which user/process is making disk utilization high.

I have orahome in vg00. Now how to justify that its the oracle and nothing st system level

Re: Higheset disk utilizaing users

Spark_2 — Thu, 12 Nov 2009 09:33:29 GMT

As it is the internal disks which have 7 standard and oracle mount points too...so becoming little difficult...fuser and touch also not helping much.

Although the above two can help in a little different scenario. Thank you

Still if someone has faced similar situation, please do guide me a bit

/dev/vg00/lvol1 1025617 166195 756860 18% /stand
/dev/vg00/lvol7 20971520 12365408 8606112 59% /var
/dev/vg00/lvol6 8290304 2660592 5585776 32% /usr
/dev/vg00/lvol5 10485760 590600 9819576 6% /tmp
/dev/vg00/orahome 6160384 4498245 1558257 74% /orahome
/dev/vg00/oraclient
6160384 19568 5757983 0% /oraclient
/dev/vg00/lvol4 10485760 5630688 4817168 54% /opt
/dev/vg00/lvhome1 2097152 1132020 905657 56% /home1
/dev/vg00/lvol8 3145728 1205880 1924712 39% /home

Re: Higheset disk utilizaing users

Michael Steele_2 — Thu, 12 Nov 2009 11:04:56 GMT

Sigh.

The problem is you only have a couple of disks and everything is on them.

I can only offer you my experience at this point, the O/S file systems will always create the greatest amount of I/O traffic on any disk. For example, you have 10 disks and the O/S, vg00, only resides on the first two. These two will have more I/O traffic than any of the other 8.

Regardless, the way to fix this problem hasn't changed in a decade, add more disks and move your applications and database out of vg00 and off of the O/S disks.

The O/S should always have their own disks and always be the only thing in vg00.

Re: Higheset disk utilizaing users

Spark_2 — Thu, 12 Nov 2009 11:13:21 GMT

Yeah Michael...I agree

so, till the movement, there doesnt seem anyway to write to Oracle team with some proofs...that u r the one utilizing highest :-(

Re: Higheset disk utilizaing users

Hakki Aydin Ucar — Thu, 12 Nov 2009 11:40:44 GMT

>Spark: The script seems to be pulling biggest files.

OK, then you can try this partition based version :

#!/bin/sh
# spacehogs-bypartition HP-
# For Linux use /bin/mount | awk '$5 != "nfs" { print $3 }'.

for p in `/usr/bin/cat /etc/fstab | awk '$3 != "nfs" { print $2 }'`
do
echo "Usage on partition $p"
find $p -xdev -type f -exec /bin/ls -ls {} ';' | awk '
{ using[$4] += $1 }
END { for (name in using)
{ print using[name], name }}' |
sort -nr |
head
done
exit 0

Re: Higheset disk utilizaing users

Spark_2 — Thu, 12 Nov 2009 12:07:36 GMT

But we can identify highest CPU and Memory users....correct?...there are those unix95 commands...can someone please paste them

and also something about disk issue

Re: Higheset disk utilizaing users

Johnson Punniyalingam — Thu, 12 Nov 2009 13:00:53 GMT

Hi Spark,

please check attached file

# UNIX95= ps -ef -o pid,sz,vsz,args

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1353245

# sar -d 5 50
-- What is the disk I/O load?

Re: Higheset disk utilizaing users

Johnson Punniyalingam — Thu, 12 Nov 2009 13:07:34 GMT

$ UNIX95= ps -efo vsz,ruser,pid,ppid,args | sort -rn | more

Re: Higheset disk utilizaing users

Michael Steele_2 — Thu, 12 Nov 2009 13:22:54 GMT

UNIX95 is an excellent tool for analyzing processes. However, it relys on the -o options found in the 'ps' command and the 'ps' command neither measures I/O or disk utilization.

I do think its worth your while using the vsz and pcpu, state, wchan, args, pid, ppid arguements though. And becomming familar with the biggest users of virtual memory on CPU time. State will be run, sleep, wait, etc. Whan will till you what pid is holding up a waiting process. All are good metrics.

Re: Higheset disk utilizaing users

Bill Hassell — Thu, 12 Nov 2009 14:02:38 GMT

Because there are so few disks on this system, the issue is very likely an Oracle configuration issue and possibly some very poor database procedures. For good performance, you need a lot of RAM and have Oracle configured to use a large SGA to cache disk activities. Setting up SGA requires a good Oracle DBA with performance tuning skills. The DBA should also be able to identify inefficient SQL statements as well as queries that use sequential searches (which means very busy disks).