- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: Extensive Constant I/O around 100%
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2008 03:40 AM
04-16-2008 03:40 AM
We are facing I/O Issues on one of our production servers.
No matter, oracle database is running or down, extensive constant I/Os were observed (around 100% disk utilization) on the Physical Volumes presented through SAN.
Please find the attached logs.
The PV c14t0d1 & c14t0d3 were presented through SAN.
Regards,
Muhammad Ahmad
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2008 03:59 AM
04-16-2008 03:59 AM
SolutionI would say this:
what is on those two disks? What processes are performing I/O?
Lets say there is a filesystem called /clutz on the disk.
fuser -cu /clutz
This will identify the processes with open file handles on the disk.
Lets say there is a process number 4433 on the disk.
ps -ef | grep 4433
That will get you process information.
It could be a number of things, but I suspect either swap because the system is busy, or a poorly written application causes the problem.
SEP
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2008 04:26 AM
04-16-2008 04:26 AM
Re: Extensive Constant I/O around 100%
I see nothing of concern in these outputs. The %busy column in sar -d simply shows the % of time during the interval that the disk was doing an IO operation. This doesn't necessarily imply an IO issue (particularly if the LUN in question is in fact on a disk array and is made up of many physical spindles).
The important numbers to look at on your sar -d output are avque, avwait and avserv. In your case these do not look partilcularly concerning. Typically if we see avque values of >2 and avserv values of >10ms we might have soemthing to be concerned about. Looking at your output I'm guessing that c0t6d0 and c3t6d0 are your system disks, and even though they have service times just over 10ms they aren't doing a significant amount of IO anyway. Your other (presumably database) disks are doing a significant amount of IO but queue lengths and service times are not anything to be concerned about.
Do you actually have a performance issue reported to you, or are you just thinking there is cos of the 100% util number?
HTH
Duncan
I am an HPE Employee

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-16-2008 05:34 AM
04-16-2008 05:34 AM
Re: Extensive Constant I/O around 100%
if you run "glance -i", you may also get some indication about the "Top disk user" and thereby investigate further.
e.g. (extract of glance output)
--------------------------------------------------------------------------------
IO BY FILE SYSTEM Users= 2
Idx File System Device Type Logl IO Phys IO
--------------------------------------------------------------------------------
1 / /dev/vg00/lvol3 vxfs 0.1/ 3.6 1.1/ 1.6
2 /stand /dev/vg00/lvol1 hfs 0.0/ 0.0 0.0/ 0.0
3 /var /dev/vg00/lvol8 vxfs 0.7/ 0.6 2.6/ 2.1
4 /usr /dev/vg00/lvol7 vxfs 0.0/ 6.4 0.0/ 0.8
5 /tmp /dev/vg00/lvol4 vxfs 0.0/ 0.0 0.7/ 0.4
6 /prod */vgcmtapp/lv_cmtapp vxfs 0.0/ 0.0 0.0/ 0.0
7 /opt /dev/vg00/lvol6 vxfs 0.0/ 2.0 0.0/ 1.8
8 /home /dev/vg00/lvol5 vxfs 0.0/ 0.0 0.0/ 0.0
9 /d10 */lv_pfsdb01 vxfs 0.0/ 0.0 0.0/ 0.0
10 /d02 /dev/vg02/lvol1 vxfs 0.0/ 0.1 0.0/ 0.0
Top disk user: PID 0, swapper 2.1 IOs/sec S - Select a Disk
hope this helps too!
kind regards
yogeeraj
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-17-2008 12:43 AM
04-17-2008 12:43 AM
Re: Extensive Constant I/O around 100%
first of all thanks a log to all.
Actually, Its an insert-intensive application (bulk inserts from 8 to 10 sources in parallel) in conjunction with fetch/select queries.
Hi SEP,
its the oracle user and oracle application processes,which are doing I/O.
Hi Duncan,
yes, the performance issue was observed when select statements are given while the parallel bulk inserts are running.
Hi Yougeeraj,
glance shows disk utilization around 100%, and memory plus cpu around 55-80%.
Any other areas to investigate?
Thank you.
Regards,
Muhammad Ahmad.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-17-2008 08:42 AM
04-17-2008 08:42 AM
Re: Extensive Constant I/O around 100%
have you use dd command to check disk utilization. the command is . . .
# dd if=/dev/rdsk/c93t9d5 of=/dev/null bs=64k
44132+0 records in
44132+0 records out
If this command return any error, that meanse the disk have a problem( effected disk).
Second thing, are you using striping or not? if not then you should implement it to reduce this type of problem.
Regards,
Piyush Mathiya
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-18-2008 08:43 AM
04-18-2008 08:43 AM
Re: Extensive Constant I/O around 100%
And was your sar output actually taken when this problem was occurring?
Because as I said I see *no* issue with the performance of these two disks.
HTH
Duncan
I am an HPE Employee

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-18-2008 09:22 AM
04-18-2008 09:22 AM
Re: Extensive Constant I/O around 100%
>>No matter, oracle database is running or down, extensive constant I/Os were observed (around 100% disk utilization) on the Physical Volumes presented through SAN.
You responded to SEP stating Oracle was up and running, using these disks.
Your opening statement leads us to believe that 100% util is seen even when Oracle has been stopped.
so, are we looking to anylyze the performance of these disks or attempt to find out whey they are executing IOs even when the app is down ???
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
04-18-2008 09:23 AM
04-18-2008 09:23 AM
Re: Extensive Constant I/O around 100%
>>No matter, oracle database is running or down, extensive constant I/Os were observed (around 100% disk utilization) on the Physical Volumes presented through SAN.
You responded to SEP stating Oracle was up and running, using these disks.
Your opening statement leads us to believe that 100% util is seen even when Oracle has been stopped.
so, are we looking to analyze the performance of these disks or attempt to find out whey they are executing IOs even when the app is down ???