Lesser evil - paging or buffer cache miss?

[Dan_173]

It has been suggested to me that decreasing an over-sized buffer cache (11.0) may resolve paging but simply pushes the I/O downstream to a buffer cache miss and subsequently a physical I/O to satisfy the cache miss - effectively only relocating the physical I/O.

However, I believed (rightly or wrongly) that PAGEOUTs were far worse than a cache miss.

What do you think?

[Sridhar Bhaskarla]

I posted this message in one of the 5 threads and it does not appear anymore.

(

Hi Dan,

There are atleast 5 posts on the same subject. I hope this thread will survive the cleaning.

The answer is 'it depends'. What do you have on the system?

If it oracle database, then there is no point in keeping more than 300-400MB of buffer cache. Oracle does it own buffering. And moreover if your system does a lot of synchronous writes, your buffer cache will be useful for only read-aheads. If your disk system also has it's own cache, then I wouldn't be much worried about cache misses on the system. I will keep an eye on sar and sar -d ( or iostat) and see if the system is waiting on IO.

On the other hand, paging out is dangerous. With cache misses, there is a chance of system being impeded. But with pageouts, we are sure that the execution times will increase as the pagedout pages will have to be paged in again. The next level is swapouts/deactivations and let us not go there.

)

Also, it will be helpful if you can post the outputs of the following when your system is busy

swapinfo -mat
sar 5 12
sar -d 5 12
sar -b 5 12
sar -q 5 12
vmstat 5 5


-Sri

[Michael Steele_2]

Normally, when a process is deactivated its because it completed its work and its state changes from running.

When a running process can't complete its work and is deactivated via swapping its real bad. Its worse if the swapped process's state changes from running to wait or sleep, for example its waiting on I/O.

The only way you stop pageouts is by reducing the load on the server (* killing processes *) or by adding more physical memory.

Regarding "...decreasing an over-sized buffer cache (11.0) may resolve paging but ..."

I don't know how much physical memory you'll gain but on paper it seems worthy. How did you conclude the 'over-sized buffer cache' on HP-UX 11.00?

Measure cache with 'sar -b 5 5'

Change with 'dbc_min' and 'dbc_max'.

What are your 'dbc_min' and 'dbc_max'?

[Bill Hassell]

Easy answer: pagouts are a serious problem for active programs. Not only must the paged process be stopped (deactivated), but it must be written to disk as a kernel task that consumes a lot of system overhead, then returned back to RAM once the process becomes a high priority--huge amounts of overhead over a very long time period. A buffer cache miss changes an I/O from memory-based to a filesystem I/O request which is significantly faster than a pageout/pagein. A low buffer cache hit rate (50%) might reduce overall throughput by 2:1 but paging can reduce throughput (for several programs in some cases) by 100:1.

Be careful that pageouts are not for memory mapped files but for actual processes. This can be deduced when the buffer cache is not at the lowest setting and still there is some paging. Or look to see that only interactive processes are in the swapped state (ps -efl | sort -rn).

[A. Clay Stephenson]

This is a no-brainer. Pageout's are far,far worse by orders of magnitude. I suppose one of the easists ways to examine this is to consider two large processes, A & B. - both are too large to "fit" in memory simultaneously. This is the classic scenario for thrashing in which each process gets only a small timeslice and then has to make roiom for the other. By comparison, buffer cache misses are expected and completely normal.

Surprisingly, there is even a downside when the buffer cache gets so big even with no memory pressure. The time required to search the buffer cache can exceed the time required to do a physical I/O operation - especially in the case where like virtually all modern disks there is on-board cache on the disk (or disk array) itself. For 11.11, most boxes peak out somewhere around 800-1200 MB of cache and 11.0 is typically half of that - your mileage will vary.

These are the kinds of questions that a good cheap sandbox is made to answer. There is no substitute for doing these measurements yourself.