Address translation faults (vmstat command)

[Carles Viaplana]

Hello,

I executed vmstat and I got this output:

# vmstat 1 4
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
3 1 0 80788 6862 2239 209 105 0 0 0 19 1619 38860 885 42 19 39
3 1 0 80788 6936 3587 363 148 0 0 0 0 1937 25867 563 46 31 23
3 1 0 80788 7083 3544 367 149 0 0 0 0 1921 26302 548 47 34 19
3 1 0 80788 7067 3444 356 145 0 0 0 0 1886 25858 558 48 25 28
#

I read "at" field in the vmstat report is really referring to the number of page faults, a virtual memory system metric, so values we have are not good.

Can I perform any tasks to improve this values?
Should I reboot system?

Note that OS is B.11.11.

Thanks in advance for your help.
Regards,

Carles

[Ivan Ferreira]

Maybe I'm a bad administratior, but if I don't see page in and page outs, I don't worry about memory, page outs indicates that the system is paging or swapping, this indicates a memory shortage.

[Carles Viaplana]

Hi Ivan,

Thanks for your comment.

In fact, I haven't problems with page-in page-out. Problem is with address translation faults (at column).

I think address translation faults could be root cause that some applications run slowly and I want to know if I'm right and how can improve system performance (it means to get 0 address translation faults).

Regards,

Carles

[A. Clay Stephenson]

You are never going to get 0 page faults and the reason is quite simple. That value actually relates to the size of the TLB (Translation Lookaside Buffer). The TLB is a very fast buffer and serves as an array of pointers to memory pages but this array is of finite size. In later versions of the PA-RISC processors it's larger but still finite. The "address translation" faults you are seeing simply indicates that the TLB is full so that one of the address entries will need to be remapped.

[Carles Viaplana]

Hello again,

I've another server and when I execute vmstat command I get this output:

# vmstat 1 10
procs memory page faults cpu
r b w avm free re at pi po fr de sr in sy cs us sy id
2 0 0 129468 1986 697 0 14 3 0 0 55 1250 3822 615 15 10 74
2 0 0 129468 1943 642 1 9 0 0 0 0 1031 3240 489 8 4 88
2 0 0 129468 1943 685 0 9 0 0 0 0 945 3052 436 8 4 88
2 0 0 129468 1943 719 0 8 0 0 0 0 912 3052 439 7 22 71
2 0 0 129468 1959 745 0 8 0 0 0 0 828 2905 401 12 8 80
2 0 0 129468 1959 767 0 7 0 0 0 0 797 2816 384 12 2 86
2 0 0 129468 1959 784 0 7 0 0 0 0 761 2709 356 8 4 88
2 0 0 129468 1943 798 0 7 0 0 0 0 729 2631 327 8 5 87
2 0 0 129468 1943 809 0 7 0 0 0 0 807 2711 351 7 6 87
2 0 0 129468 1943 818 0 7 0 0 0 0 760 2687 332 8 26 66
#

As in this case I usually have 0 errors I assume something was wrong on 1st server.

Then, I could say:

- This can't be root cause of some applications run slowly
- It's normal this value isn't 0

BTW, can we increase TLB?

Thanks in advance for your help.

Regards,

Carles

[Hein van den Heuvel · Accepted solution]

"Address translation faults" will happen. No big deal.

However, your vmstat does suggests problems in this area.
Your User:System ratio is only 2:1
This is not great. It may me ok for your application, but it could also easily be due to excessive page faulting

Your Active:Free memory ratio is more than 10:1
So there is very little (memory) room left to play.
This could easily be explained by a dbc_max, and that could be fine.
However, under 10% memory free could also be a sign of over-committed memory.

Besides system configuration like dbc settings, are there any major knobs to turn in your application like a database buffer cache?
Ipcs -m might just give a starting point.

Does vmstat -s give more insight?

Finally, are you using 'large pages' for your major images? Check out the man pages for chatr -pd and -pi

Be sure to google around for this subject.
Lots of cookbooks and hints have been published.

http://h21007.www2.hp.com/dspp/files/unprotected/devresource/Docs/TechPapers/UXPerfCookBook.pdf

For example an older but valid page on memory in "ORACLE for UNIX, Performance Tuning Tips, Part No. A22535-2" which can be foudn for example as: http://www.allenhayden.com/cgi-bin/getdoc.pl?file=oraunix.pdf

Hth,
Hein

[A. Clay Stephenson]

The only way to increase the size of the TLB is to change processors; it's a hardware device.

[Carles Viaplana]

All your comments and docs you sent are helpful for me.

Thanks again to all.

Regards,

Carles