What REALLY is PAGEOUT on vmstat command output

[Steve Post]

Most books I find say that the vmstat command shows page in's in the pi column, and page out's in the po column. There is also a de column sometimes, that is called desperation swapping. This de column should be 0.
The po column should normally be 0. The pi column should have some numbers in it, but should not be high.

Ok. I can see numbers on the vmstat command. But I do NOT see the real meanings behind these terms. Supposedly when a process starts, disk data gets "paged into memory". When it is done, it gets "paged out" back to disk. This infers that "paging in" means reading disk, and "paging out" means writing to disk. But "page ins" do not equal "page outs". There are hardly any page outs. Does this mean the memory hardly ever writes back to disk? I find that hard to believe. But why did vmstat call it page out? Why does the term "page out" mean write back to disk?

So this leads me to ask......

What do these pi, po terms REALLY mean for the vmstat command?

[Court Campbell · Accepted solution]

The system bring pages into memory when what it is for is nor already in memory. It will write pages to disk when memory use is high.

So usually a program might request some data. It will check memory, it the data is there it will read it into the cpu registers and do it's thing. If not it will page fault and bring the pages it need into memory. It will then work with those pages. (That is a shortened version of what happens.)

When programs need more memory than what is actually physically available is when you will start paging out. You have to free up some memory in order to read new pages in. So the memory manager basically decides what can be paged to disk temporarily so that new pages can be read into memory. Later the pages will be read back into memory when they are needed again.

>> Does this mean the memory hardly ever writes back to disk?

I think you are confusing memory management with system writes to disk. They are not the same thing.

[Michael Steele_2]

Paging is taking a section of contiguous address from RAM to disk.

Paging in is the anticipation or prefetching of address and does not count as a problem.

Paging out is unanticipated and counts as an error.

[Steven E. Protter]

Shalom,

What do these pi, po terms REALLY mean for the vmstat command?

It means processes are being paged in or out of swap.

Higher numbers are bad and indicate a performance problem.

SEP

[Court Campbell]

Maybe this will help:

http://docs.hp.com/en/5965-4641/ch01s10.html?btnNext=next%A0%BB

[Steve Post]

I'm going to try an analogy!

You have a restaurant with a slightly goofy head waiter. Everyone that is showing up for a seat is giving him a bribe. So he takes the money and seats them (no matter what). The groups of people are processes. The individual people are "pages" of memory. And the restaurant is the memory of the computer.

A party comes in. A Mr. Process, party of five. Mr Process asks for a table, the waiter finds a table with 5 chairs and lets him in.

A little later on, the restaurant is full. Now a Mr. VIP comes in with a request for just two. The waiter knows there are no free chairs but goes in search of some way to satisfy Mr. VIP.

He finds a young couple that is talking romantic over dessert.
Waiter: "HIT THE ROAD."
Romantic couple: "But we are in the middle of our dessert!"
Waiter: "OK. We have a little stall in the kitchen we use to hold onto people's unfinished food. It is called Kitchen-A-la-SWAPSPACE." He whisks the couple to kitchen with their desserts.
He then quickly leads Mr. VIP to the newly acquired table.
This Mr. VIP is one process, with 2 pages, and it is a page in.
But the young couple that got kicked into the kitchen is a page out. Am I Right?

[Michael Steele_2]

Yes. The visit from the VIP is unanticipated, (* he doesn't have a reservation and isn't standing inline *).

[Dennis Handly]

>when a process starts, disk data gets "paged into memory". When it is done, it gets "paged out" back to disk.

No, when the process is done, the data vanishes. Only when some other process needs the space, is it paged out.

>But "page ins" do not equal "page outs".

Naturally. You could page in multiple times if only reading a page. Of course you may have to page some other process out to free space.

>Does this mean the memory hardly ever writes back to disk?

Yes. The data is read in at program start and hopefully there was no reason to ever write it back to swap, plenty of memory.

>But why did vmstat call it page out? Why does the term "page out" mean write back to disk?

It is removing the page from memory and putting it out to pasture/swap.

>Waiter: "We have a little stall in the kitchen we use to hold onto people's unfinished food. It is called Kitchen-A-la-SWAPSPACE." He whisks the couple to kitchen with their desserts.

You forgot to mention that the waiter shoots the couple with a dart gun so they go to sleep. ;-)

[Venkatesh BL]

> He finds a young couple that is talking romantic over dessert.

Well, in actual case, it happens when the couple fall asleep waiting for the dessert to arrive!...(meaning no action for a long time.)

[Hakki Aydin Ucar]

I want to share my vmstat notes below:

Virtual Memory Activity runs the vmstat command.
vmstat provides virtual memory statistics. It provides information on the status of
processes, virtual memory, paging activity, faults, and a breakdown of the percentage of
CPU time.
In the following examples, the output was produced nine times at five-second:
intervals. The first argument to the vmstat command is the interval; the second is the
number of times you would like the output produced:
# vmstat 5 9
You certainly get a lot for your money out of the vmstat command. Here is a brief
description of the categories of information produced by vmstat.
Processes are classified into one of three categories: runnable ("r"), blocked on I/O or
short-term resources ("b"), or swapped ("w").
Next, you see information about memory. "avm" is the number of virtual memory pages
owned by processes that have run within the last 20 seconds. If this number is roughly the
size of physical memory minus your kernel, you are near forced paging. The "free" column
indicates the number of pages on the system's free list. It doesn't mean that the process is
finished running and these pages won't be accessed again; it just means that they have not
been accessed recently. I suggest that you ignore this column.
Next, is paging activity. The first field ("re") shows the pages that were reclaimed. These
pages made it to the free list but were later referenced and had to be salvaged.
Next you see the number of faults in three categories: interrupts per second, which usually
come from hardware ("in"), system calls per second ("sy"), and context switches per second
("cs").
The final output is CPU usage percentage for user ("us"), system ("sy"), and idle ("id"). The
nice processes are lumped together with user processes in this output.
If you run an I/O-intensive workload, you may indeed see a much activity in runnable
processes ("r"), blocked processes ("b"), and the runnable but swapped ("w") processes. If
you have many runnable but swapped processes, you probably have insufficient memory.
If you see a lot of waiting on I/O, you probably have an I/O bottleneck.

Addition: The po column is the most important in terms ofpaging: it indicates the number of page-outs and should ideally be very close to zero. If it isn't, processes are contending for the available memory and the system is paging. Paging activity is also reflected in significant decreases in the amount of free memory (fre) and in the number of page reclaims (re)â memory pages taken away from one process because another one needs them even though the first process needs them too.

[Bill Hassell]

> What do these pi, po terms REALLY mean for the vmstat command?

pi = number of pages read in from disk. VERY IMPORTANT: page in counts both page back from swap space *and* starting a program for the first time. Both are page in events. Very high page in rates (hundreds) is normal.

page out = write pages of a deactivated program to swap space. This is truly a system that is low on RAM and must make room for more processes.

> It means processes are being paged in or out of swap.

No, page in is whenever a program starts or is returned from swap. These are very different conditions but counted with the same pi counter.

> Higher numbers are bad and indicate a performance problem.

Only po matters. pi can be large on a busy system as it represents program starts. po is also confusing because it measures both paging out to swap space but also count memory-mapped file access. There is no way to tell the difference other than swap space is unused (see: swapinfo -tam) and po is greater than 0.

2 digit numbers 10-99 in po indicate a fair amount of paging and lack or RAM, but performance may be acceptable. 3 digit (100+) indicates serious memory shortage and severe degradation of the entire system. Page out affects everything, especially in a multi-processor system. when memory must be rearranged, a spinlock occurs that prevents all the other processors from making any changes to RAM and this can affect other processes.

There are exactly two fixes for high pagew out:

1. Add a lot more RAM (2-3 times more RAM).

2. Stop running so many processes at the same time.

[Steve Post]

Thank you ALL. It's nice to see clean, clear explanations.