- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: troubleshooting disk io
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-12-2011 08:45 PM
11-12-2011 08:45 PM
troubleshooting disk io
Hello,
i'm troubleshooting some performance issues with an application, so far i haven't found the problem and suspect its the application but did notice some unexpected disk IO problems.
It's an integrity service gaurd cluster running 11.31.
CPU and Load Average seem OK e.g. 40-50% idle cpu 1-2 load average.
using sar -d i've noticed the following:
the avwait spikes to 40 or so which outweighs the avserv now it doesn't go to 100% busy at the time and when this happens it happens for the san disks as well.
Now i've been using the application during these hits and it runs OK. My thougts are to get a performance hit on the application it would have to be prolonged high IO results not a quick spike? I've been running sar and by the time the next round of results come back the spike is gone.
the root disk is an lvm mirrored disk, the only thing on the root volume is the hpux OS and everything else has been moved to the san.
Why would i be seeing large avwait times on the root disk or is this normal?
also sar -b seems to be high on the write cache the average is around 40 but i've had it go lower than that. Althought i'm not sure its calculation averages properly.
I've read that its best to keep wcache above 70?
Any help and suggestions would be great.
Like i said this may be normal i'm just chasing performance issues so digging at everything at the moment.
Thank you.
- Tags:
- sar
- Serviceguard
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-14-2011 12:45 AM
11-14-2011 12:45 AM
Re: troubleshooting disk io
>> Why would i be seeing large avwait times on the root disk or is this normal?
just a while guess, do you have large swap space in your root disks?
Could you post your vmstat and sar or glance output?
//
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-14-2011 02:06 AM
11-14-2011 02:06 AM
Re: troubleshooting disk io
Hi Thanks for replying.
Good point, i didn't even think of that.
Yes we do its 24GB.
Here are some stats, i've included Glance because i noticed its constatly blocked on Terminal and Semphore?
sar -d
19:55:01 c2t0d0 10.60 0.50 24 702 0.00 6.37
c9t0d4 0.20 0.50 7 246 0.00 0.49
c7t0d4 0.20 0.50 7 245 0.00 0.59
disk17 0.20 0.50 13 491 0.00 0.54
disk18 15.40 0.50 28 721 0.00 9.17
disk19 10.60 0.50 24 702 0.00 6.37
19:55:06 c2t0d0 28.20 7.36 103 10866 17.26 14.62
c9t0d4 2.60 0.54 30 2680 0.29 5.60
c7t0d4 2.60 0.53 30 2659 0.61 5.68
disk17 5.40 0.53 60 5339 0.45 5.64
disk18 38.00 7.49 115 10916 20.22 17.13
disk19 28.20 7.36 103 10866 17.26 14.62
vmstat
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
22 1 0 3326228 858410 0 0 0 0 0 0 0 5017 194314 9338 16 14 70
Glance (B)
IPC 0.2 40.78 7.0 Cache 0.0 1.56 0.3
Job Control 0.1 23.32 4.0 CDROM IO 0.0 0.00 0.0
Message 4.8 815.50 140.1 Disk IO 0.0 0.00 0.0
Pipe 0.6 99.88 17.2 Graphics 0.0 0.00 0.0
RPC 0.0 0.00 0.0 Inode 0.0 0.00 0.0
Semaphore 0.0 0.00 0.0 IO 0.1 17.61 3.0
Sleep 81.2 13890.84 2386.7 LAN 0.0 0.00 0.0
Socket 1.5 248.71 42.7 NFS 0.0 0.00 0.0
Stream 0.1 17.24 3.0 Priority 0.0 8.45 1.5
Terminal 0.0 5.82 1.0 System 1.7 291.25 50.0
Other 9.6 1636.36 281.2 Virtual Mem 0.0 1.53 0.3
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-15-2011 05:42 AM
11-15-2011 05:42 AM
Re: troubleshooting disk io
OK so you can see i'm not paging out so probably not a huge concern about the swap at the moment.
I did find one interesting setting
the fcache_fb_policy as been set to 1 enabling it, i'm not sure why this has been set. I checked write_throttle and it hasn't been set.
anyone had anything to do with the fb_policy?