rx6600 performance issue

[Nyck_1]

We have an rx6600 running 11.23 and its been working fine for months. Yesterday afternoon it became very sluggish and non-responsive. When I ran a top or sar the sys usage was through the roof:-

the SYS usage report from top was hitting 80%+ across all 8 cpu's

In the end I had to reboot the box and this took about 1.5 hours to do so. It was hanging on kcmodule which was hitting the system hard, in the end I ran a kill -9 against the process and then it finally rebooted and came back up fine.

What kind of issues would cause this high SYS load?

The server has 64GB of ram and does have oracle running on it.

[Duncan Edmonstone]

Nyck,

> What kind of issues would cause this high SYS load?

All sorts of issues - usual culprit is bad application behaviour. From the data you provided it's reall not possible to say.

> When I ran a top or sar the sys usage was through the roof

So in top, did you note *which* processes were using all this SYS CPU time?

>It was hanging on kcmodule which was hitting the system hard

What does that mean? Do you mean the reboot hung at one of the calls to the kcmodule command during shutdown and didn't seem to get any further, or do you mean it was the kcmodule call which was using all the SYS CPU?

Diagnosing these sorts of issue after the fact is difficult (think air accident investigation, and you might not find the Closed box!)

First off, it would be interesting to see if there is any useful pointers in the syslog from before the reboot - looking at /var/adm/syslog/OLDsyslog.log would be a good start...

Second, do you have the Performance Agent for this system? That *may* have recordede some usefull data - what does:

ovpa status

return - that will tell us if you have OVPA installed or not...

HTH

Duncan

[Suraj K Sankari]

Hi,

As Duncan said there are many reason why this happened, if you have any monitoring system then check when the load goes high and which process was running at that time. Moreover you can check Oldsyslog.log file...

Suraj