CPU & Memory Utilisation.

[Jimmy Page]

I'm trying to see if the sluggish performance of a batch suite is down to the server config or just some poor code. I've been getting some details via 'Patrol' and I have a couple of questions on the results:
1: CPU. The CPU utilisation occaisionally hits 100% for a short time. Most of the time it peaks at 90%. Question - is there a degradation in performance while the usage climbs towards 100%? eg: is a system with CPU usage at 80% performing worse than a system at 40% usage?

2: Memory. The stats are reporting that the memory is pageing out about 15-50 pages a second. However, there is always at least 500000 pages of memory being reported free too. Am i right in assuming that the pageing out is (probably) not a performance issue and is simply processes finishing and being 'flushed' from mem?

Any help would be appreciated.

thanks.

[Geoff Wild · Accepted solution]

Check the output of vmstat

Say, vmstat 5,5

Are you getting a lot of pi's /po's?

Is the sr (scan rate) high? (above 0)

If it is, then you are swapping - and you probably need more memory...CPU will go up if the scanrate is high....

Rgds...Geoff

[Jimmy Page]

thanks geoff

Scan rate is 0.
Page out = apprx 30
page in = 1 or 2.

Does this mean that the system isn't swapping? And therfore is performing ok within it's resources?

[Bryan D. Quinn]

Hello,

To add to what Geoff said, check top -s 1. Where is vhand? If swapping is happening then vhand is working harder. The harder vhand works, the more performance degradation will occur. If you are swapping hard then vhand will be eating up your CPU. I have had it happen on one of my servers to the point of the system freezing. Just remember if vhand wants more cpu it will get it. That is why you do not want your system swapping.

hope this helps!
-Bryan

[Patrick Wallek]

If you are seeing 'po' in vmstat then chances are you are swapping, which is really bad for performance. What does 'swapinfo -tam' show?

If you have glance, use it to look at the system resources and see what is running and what memory usage is.

As far as CPU usage goes, if you are not hitting 100% usage for long periods of time, then you are probably OK.

[Geoff Wild]

Paging out â poâ happens throughout the day for various mysterious OS reasons.
Presumably if we have the memory we shouldn't need to page out ever.
As processes die we free the pages and they can be used by new processes.
However we do see paging out for filesystem buffers quite a bit, so we typically ignore pages out at an average level.

Doesn't sound like you are swapping - but as
Bryan says, check vhand...

Rgds...Ge

[Geoff Wild]

Just a thought, you might want to check out:

http://www.sarcheck.com/

Really good utility - fairly expensive - but you can get a 30 day trial for free - run it on your system - it provides very good information and recommendation.

Rgds...Geoff

[Jimmy Page]

Swapinfo is showing as 44% used. Memory (via glance) is showing as 88% used. Pageing is still being shown on vmstat.
It does look very much as if the pageing out is part of the 'mysterious O/S' functions.
I didn't think it was swapping but I need a little reassurance.
Thanks everyone.

[Patrick Wallek]

If you are seeing 44% swap space usage, then you are doing more than "Mysterious OS functions" or at least were at one point.

How much RAM do you have? What applications are running?

Can you paste the output of swapinfo into your next reply?

Yes, you will see a *few* page outs as normal part of the OS, but you should *not* see 50 / second nor should you see 44% swap space usage normally.

[Jimmy Page]

swapinfo output:
$ swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 512 44 468 9% 0 - 1 64,0x030002
dev 2000 44 1956 2% 0 - 1 64,0x030008
reserve - 751 -751
memory 1526 901 625 59%
total 4038 1740 2298 43% - 0 -


vmstat output:$ vmstat 5 5
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
3 1 0 22501 89857 154 42 0 8 110 0 0 448 9407 357 18 6 75
0 1 0 21377 89803 251 55 0 6 153 0 0 489 6053 563 8 5 87
1 1 0 19598 89801 139 30 0 4 104 0 0 489 3193 332 2 1 97
5 1 0 19285 89625 240 63 0 5 143 0 0 527 7966 676 7 9 85
0 1 0 17009 89191 201 40 0 5 127 0 0 605 9960 419 10 6 83


The 50 pages a second is being reported via Patrol. I have no idea how it has come to that figure as vmstat isn't reporting anywhere near that figure.

thanks.