Lesser evil: pageouts or cache miss?

[Dan_173]

If I resolve/reduce a pageout problem by reducing my buffer cache from 1G (of 3.75) to 400M, I'm likely to increase buffer cache misses which may result in increased phys IOs downstream. Haven't I just relocated the phys IO and made nil net gain?

Or is there some overhead to PAGEOUTs which makes them an order of magnitude worse than a buffer cache miss?

Thanks in advance
Dan

[Jarle Bjorgeengen]

Hi Dan,

if you go short on memory , you risk that processes are competing for memory. This means that your pages won't only get paged out once , but you will constantly page in and out pages, because processe will try to access pages that is paged out, and vhand will struggle to find pages to page out again because you are low on free mem. This is called trashing , and you may not be able to log into your system. (Because all resource are used on paging in/out)

Low BC hit rate will give you lower i/o performance towards filesystems and block-devices, but won't put your system in a trashing state.

RGds Jarle

[Steven Gillard_2]

IMHO pageouts are much worse than cache misses. My reasoning goes something like this:

* The buffer cache is for caching of files that reside on disk anyway. A process wanting to read a file generally expects to cost a physical IO.

* A pageout is a page of virtual memory being written out to disk. Next time the process wants to access that particular part of memory it will result in another disk access. That is definitely an order of magnitude worse than a disk access for a file read!

Keeping cached file data in memory at the expense of virtual memory pages doesn't really make a lot of sense to me. 1Gig is a very large buffer cache, reducing it to 400Meg would be a wise idea.

Just my ??0.02

Regards,
Steve

[Stefan Farrelly]

Pageouts are far far worse than buffer cache misses. Once youre out of ram your server starts to slow down by a factor of around 100! If you have buffer cache misses it simply means a few more io's, nowhere near as bad.

Our main goal on all our large servers is;
Keep some Ram free (ie. NO pageouts). We never let them run out of ram due to the drastic decrease in performance that would result.

[Dan_173]

Many thanks for the feedback.
Anyone else add technical reasoning to..

"Why are pageouts so much more crippling than cache misses?"

[Stefan Farrelly · Accepted solution]

"Why are pageouts so much more crippling than cache misses?"

This is difficult to explain in a short answer but let me try.
Pageouts and indicative of memory exhaustion. This then causes the OS to pageout processes from memory to disk at a high priority, causing heavy I/O on the swap areas (usually system disks). In time all processes are accessed in a round robin fashion, which means periodically all paged processes must be paged back into memory (again, this runs at a high priority and uses heavy I/O on swap areas). If memory is still exhausted processes must again be paged back to disk. And so it continues.

Basically it causes your server to thrash. This is what causes the large performance drop.

Basically, pageouts indicate memory bottlenecks of which the major ones are;
- Saturation of memory
- A large VM queue
- Resource starvation
- user dissatisfication with response time

And straight from the HP-UX Performance Tuning book, the solution to memory bottlenecks is (in order);
- on small systems reduce the size of the kernel
- carefully reduce the size of the fixed buffer cache
- use dynamic buffer cache and tune if carefully
- reduce the use of memory locking
- etc.

[Dan_173]

Thanks Stefan. That's great.
Dan

[Michael Steele_2]

Dan,

(* didn't we just do this two days ago and obtain the .... *)

[Sridhar Bhaskarla]

Hi Dan,

//
Many thanks for the feedback.
Anyone else add technical reasoning to..

"Why are pageouts so much more crippling than cache misses?"

//

In fact you posted this question multiple times on that day and here are more responses to one of those threads on the same subject.

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xd05b0ea029a2d711abdc0090277a778c,00.html

-Sri