topic Re: Performance Issue in Operating System - HP-UX

Performance Issue

kunjuttan — Mon, 27 Dec 2010 08:48:31 GMT

Hi all,

I need your help very badly.Our application team is telling the server performance is very poor and slow.What should I do for checking the same.

Its a Superdome server.OS is HP-UX1131

Re: Performance Issue

sarfaraj ahmad — Mon, 27 Dec 2010 08:58:11 GMT

hi,
first listen the problem properly and check exactly what problem they are facing.
for performance issue there is so many things involved like CPU, Memory, disk I/O, DB, Application etc.

so better please understand the issue and if possible also compare with normal situation & gather those data.

after this we can proceed for further analysis.

Re: Performance Issue

Shibin_2 — Mon, 27 Dec 2010 08:58:33 GMT

Use glance to watch, your CPU, Memory utilization. Use, top, swapinfo, vmstat commands to find virtual memory and swap utilization.

Re: Performance Issue

Ishwar_1 — Mon, 27 Dec 2010 09:00:15 GMT

check memory, cpu, disk performance, kernel usage, zombie process, uptime and syslog for some error

Re: Performance Issue

Dennis Handly — Mon, 27 Dec 2010 13:24:15 GMT

>Our application team is telling the server performance is very poor and slow.

What does top show? How many CPUs do you have?
How much RAM/swap?
swapinfo -tam

>It's a Superdome server.

What exact model?

Re: Performance Issue

Alzhy — Mon, 27 Dec 2010 14:12:50 GMT

Please furnish captures of :

glance
glance -m
sar 5 5
sar -d 5 5
top (if you do not have glance)

when issue is happening.

We should then be able to guide you what areas to look at to address your "performance" issue.

Re: Performance Issue

kunjuttan — Mon, 27 Dec 2010 14:20:46 GMT

hi thanks....
i dont have glance installed in our box...
Its Superdome-2 server..

Re: Performance Issue

Dennis Handly — Mon, 27 Dec 2010 14:27:54 GMT

>It's Superdome-2 server.

What exact model? Use the model command.

>Ishwar: zombie process

These shouldn't have any effect on performance.

Re: Performance Issue

Bill Hassell — Mon, 27 Dec 2010 14:30:07 GMT

Generally, the system is performing at full speed, but application and database people think that the system administrator can set the kernel parameter GOFAST=1 and everything will be fine. The reality is that their app and database work OK but are very badly configured, causing unnecessary disk I/O and/or massive use of RAM which can cause swapping.

Another question is: How do they know that the server is running slow? Did it run faster in the past? If so, what was changed? If it has always run slow, what does the application and/or database vendor recommend to improve performance?

The following script shows my point:

while :
do :
done

3 lines and now CPU usage is 100% for this script. Is it running correctly? Yes. Is it doing anything useful? No. How do I reduce the CPU usage? Ans: add a sleep command:

while :
do :
sleep 1
done

Now the program consumes no meaningful amount of CPU time. And it still does nothing useful.

So the question to the application/database designers would be: Is your app/DB designed for efficiency, and what do your app/DB tools tell you about performance? You (the system administrator) could run sar -d and find huge amounts of I/O to a particular disk only to find that the DBA is reading the database serially. Does it work? Yes. Is it efficient? No.

Re: Performance Issue

Alzhy — Mon, 27 Dec 2010 14:38:50 GMT

Wow.. SuperDome-2 !
The 'Dome in a C7000 Chassis?

Are you running a DB, Web-Tier or a mixed workload?

Please give as a hint of how your server is performing:

sar 5 5
sar -d 5 5
vmstat 5 5
swapinfo -atm
top

Also, do you have HyperThreading enabled in case your workload can benefit from it?

Re: Performance Issue

Emil Velez — Mon, 27 Dec 2010 23:39:33 GMT

1. Dont believe the application people.
2. Install glance and the performance agent.. If you install it you should be able to start a 60 day trial. You can do alot of tuning of your system in 60 days and then if you like the tool (HP hopes you will) then you can decide if its worth the price and consider a purchase but until that point you have used it.

3. First look at the load average (run queue). If the average is under 1 then you do not have a cpu bottleneck. You can get this from top

4. second look at the busiest disk and if it is less than about 70% busy you probably not have disk bottlenecks.

sar -d 5 1000

this will show you all of your disks.

5. Next look at memory utilization. IF you are less than 90% full then you have enough memory.

This is only a start but some basic things to look at.

Re: Performance Issue

kunjuttan — Wed, 02 Feb 2011 15:16:28 GMT

Is glance software is a freeware or do i need license for the same??

Re: Performance Issue

Alzhy — Wed, 02 Feb 2011 15:24:00 GMT

It is licensed software but free to try for up to 120 days.

BUT let me ask you -- if this is a SuperDome i2 -- chances aer you are running better than HP-UX 11.31 Base OE -- that means your SHOULD have GlancePlus/Pak installed and alerady part of your OS...

can you see if you can luanch "glance"? Also see if there is a /opt/perf tree. Your Glance Executable should be /opt/perf/bin/glance

If you are on an X-windows environment -- then it will launch it gaphically with xglance.

Re: Performance Issue

Duncan Edmonstone — Wed, 02 Feb 2011 16:04:02 GMT

Alzhy,

> The 'Dome in a C7000 Chassis?

Whilst the SD2 certainly borrows a lot of its technology from a C7000 chassis, it definately isn't a C7000 chassis!

The C7000 chassis is 10U in height, the SD2 chassis is 18U in height. You can't put SD2 cell-blades in a C7000, and you can't put standard blades from a C7000 in a SD2 chassis (at least not yet... maybe never)

But I'm sure you know this..

Duncan

Re: Performance Issue

Ismail Azad — Wed, 02 Feb 2011 16:30:02 GMT

Hi kunjuttan,

Yes glance is free but it depends on your OE as one of the needs of an operating environment is for "license management". When you talk in terms of OEURs as alzhy said glance is there on base OE. However if you are having an O.S of an initial release which had OES like foundation, enterprise, M.C.O.E etc.. The foundation OE does not have glance. But all operating environment update releases {base, HA, VSE, DCOE } have glance.

Regards
Ismail Azad

Re: Performance Issue

Duncan Edmonstone — Wed, 02 Feb 2011 18:03:56 GMT

Ismail,

No, the Base OE (BOE) _does not_ contain Glance. Only VSOE, HAOE and DCOE have Glance.

In "old money" (i.e. the old operating environments), FOE did not have glance, whilst EOE and MCOE did.

Kunjuttan...

swlist *OE

will tell you which operating environment you have installed.

HTH

Duncan

Re: Performance Issue

Dennis Handly — Wed, 02 Feb 2011 20:47:05 GMT

>Duncan: swlist *OE

You should quote that to be safe: swlist "*OE"

Re: Performance Issue

Jean-Luc Oudart — Thu, 03 Feb 2011 11:07:49 GMT

Hi

When the application team is telling the server performance is very poor, what are they comparing with ? Do you have a baseline performance ? Is this new behaviour ? What changes have been introduced to the system prior to the change in performance ?

Also, I believe we could help better if you could tell us more about the application (underlying sw, database engine, ...) and infrastructure (eg SAN storage).

Regards
Jean-Luc