Performance Issue

[kunjuttan]

Hi all,

I need your help very badly.Our application team is telling the server performance is very poor and slow.What should I do for checking the same.

Its a Superdome server.OS is HP-UX1131

[sarfaraj ahmad]

hi,
first listen the problem properly and check exactly what problem they are facing.
for performance issue there is so many things involved like CPU, Memory, disk I/O, DB, Application etc.

so better please understand the issue and if possible also compare with normal situation & gather those data.

after this we can proceed for further analysis.

[Shibin_2]

Use glance to watch, your CPU, Memory utilization. Use, top, swapinfo, vmstat commands to find virtual memory and swap utilization.

[Ishwar_1]

check memory, cpu, disk performance, kernel usage, zombie process, uptime and syslog for some error

[Dennis Handly]

>Our application team is telling the server performance is very poor and slow.

What does top show? How many CPUs do you have?
How much RAM/swap?
swapinfo -tam

>It's a Superdome server.

What exact model?

[Alzhy]

Please furnish captures of :

glance
glance -m
sar 5 5
sar -d 5 5
top (if you do not have glance)

when issue is happening.

We should then be able to guide you what areas to look at to address your "performance" issue.

[kunjuttan]

hi thanks....
i dont have glance installed in our box...
Its Superdome-2 server..

[Dennis Handly]

>It's Superdome-2 server.

What exact model? Use the model command.

>Ishwar: zombie process

These shouldn't have any effect on performance.

[Bill Hassell]

Generally, the system is performing at full speed, but application and database people think that the system administrator can set the kernel parameter GOFAST=1 and everything will be fine. The reality is that their app and database work OK but are very badly configured, causing unnecessary disk I/O and/or massive use of RAM which can cause swapping.

Another question is: How do they know that the server is running slow? Did it run faster in the past? If so, what was changed? If it has always run slow, what does the application and/or database vendor recommend to improve performance?

The following script shows my point:

while :
do :
done

3 lines and now CPU usage is 100% for this script. Is it running correctly? Yes. Is it doing anything useful? No. How do I reduce the CPU usage? Ans: add a sleep command:

while :
do :
sleep 1
done

Now the program consumes no meaningful amount of CPU time. And it still does nothing useful.

So the question to the application/database designers would be: Is your app/DB designed for efficiency, and what do your app/DB tools tell you about performance? You (the system administrator) could run sar -d and find huge amounts of I/O to a particular disk only to find that the DBA is reading the database serially. Does it work? Yes. Is it efficient? No.

[Alzhy]

Wow.. SuperDome-2 !
The 'Dome in a C7000 Chassis?

Are you running a DB, Web-Tier or a mixed workload?

Please give as a hint of how your server is performing:

sar 5 5
sar -d 5 5
vmstat 5 5
swapinfo -atm
top

Also, do you have HyperThreading enabled in case your workload can benefit from it?

[Emil Velez]

1. Dont believe the application people.
2. Install glance and the performance agent.. If you install it you should be able to start a 60 day trial. You can do alot of tuning of your system in 60 days and then if you like the tool (HP hopes you will) then you can decide if its worth the price and consider a purchase but until that point you have used it.

3. First look at the load average (run queue). If the average is under 1 then you do not have a cpu bottleneck. You can get this from top

4. second look at the busiest disk and if it is less than about 70% busy you probably not have disk bottlenecks.

sar -d 5 1000

this will show you all of your disks.

5. Next look at memory utilization. IF you are less than 90% full then you have enough memory.

This is only a start but some basic things to look at.

[kunjuttan]

Is glance software is a freeware or do i need license for the same??

[Alzhy]

It is licensed software but free to try for up to 120 days.

BUT let me ask you -- if this is a SuperDome i2 -- chances aer you are running better than HP-UX 11.31 Base OE -- that means your SHOULD have GlancePlus/Pak installed and alerady part of your OS...

can you see if you can luanch "glance"? Also see if there is a /opt/perf tree. Your Glance Executable should be /opt/perf/bin/glance


If you are on an X-windows environment -- then it will launch it gaphically with xglance.

[Duncan Edmonstone]

Alzhy,

> The 'Dome in a C7000 Chassis?

Whilst the SD2 certainly borrows a lot of its technology from a C7000 chassis, it definately isn't a C7000 chassis!

The C7000 chassis is 10U in height, the SD2 chassis is 18U in height. You can't put SD2 cell-blades in a C7000, and you can't put standard blades from a C7000 in a SD2 chassis (at least not yet... maybe never)

But I'm sure you know this..

Duncan

[Ismail Azad]

Hi kunjuttan,

Yes glance is free but it depends on your OE as one of the needs of an operating environment is for "license management". When you talk in terms of OEURs as alzhy said glance is there on base OE. However if you are having an O.S of an initial release which had OES like foundation, enterprise, M.C.O.E etc.. The foundation OE does not have glance. But all operating environment update releases {base, HA, VSE, DCOE } have glance.

Regards
Ismail Azad

[Duncan Edmonstone]

Ismail,

No, the Base OE (BOE) _does not_ contain Glance. Only VSOE, HAOE and DCOE have Glance.

In "old money" (i.e. the old operating environments), FOE did not have glance, whilst EOE and MCOE did.

Kunjuttan...

swlist *OE

will tell you which operating environment you have installed.

HTH

Duncan

[Dennis Handly]

>Duncan: swlist *OE

You should quote that to be safe: swlist "*OE"

[Jean-Luc Oudart]

Hi

When the application team is telling the server performance is very poor, what are they comparing with ? Do you have a baseline performance ? Is this new behaviour ? What changes have been introduced to the system prior to the change in performance ?

Also, I believe we could help better if you could tell us more about the application (underlying sw, database engine, ...) and infrastructure (eg SAN storage).

Regards
Jean-Luc