Memory bus throughput

[ViS_2]

Hi!
Is there any method to find out the current memory bus loading. Something like "sar -b" or "sar -d" for buffer cache or block devices.

I have the rp7400 with 6GB of RAM installed, the 2 memory extenders inside;
HP-UX 11.0;
The oracle's database is small enough to reside in the RAM entirely, so no swapping, no disk activity occurs. But sometimes strange delays happens.

Thanks.

[Joseph Loo]

hi,

use ths command:

# vmstat 3 10
which reports virtual memory statistics of 10 3-secs interval.

man vmstat for more info.

regards.

[ViS_2]

Yes, I know about vmstat, but which numbers display that I need?

[Joseph Loo]

hi,

for vmstat, the free column shows the free virtual memory in your server. virtual memory is about 4 times less than the physical memory.

you may also use top command, where for Memory section, there is a free field which shows free physical memory.

regards.

[TommyT]

Look if po (Pages paged out) > 0 over sustained periods of time.


//tommy

[ViS_2]

I want to find out how many [MG]Bytes per second pass through the memory bus.
So, as I know the pi/po numbers illustrate the swapping activity, and as I wrote above it is absent (in my situation all page counters are zeros). The "avm" colomn demonstrates the number of active memory pages and as I understand has no concern to my question.

[ViS_2]

(in my situation all page counters are zeros)

or very close to them.

[rick jones · Accepted solution]

IIRC, there are no supported tools out there to measure memory bus utilization on systems.

There is an unsupported utility called "pi" that will retrieve CPU performance statistics. One of those gives the utilization of the bus to which the CPU is connected. That will also be the "memory" bus utilization on simpler systems, however, an rp7400 is a slightly more complicated machine - it actually has two memory busses, and depending on how you've loaded the CPUs each CPU will be on one, or the other.

So, if you happen to know which CPUs are on which bus, the "sysbus" metric/test from pi will give some idea of the memory bus loading. One systems with heirarchical memory systems (eg 'Domes and rp7410 and rp8XXX) the sysbus will not be quite as useful since it will not give visibility to the other busses.

It would be good to have a system block diagram handy while you do this :)

One other thing - as memory busses become saturated, cache misses take longer and longer to satisfy. That means CPUs sit "idle" longer and longer - however this "idle" is not "idle CPU" as in vmstat/top/glance - it actually shows-up as used CPU - the CPU isn't really "idle" but "stalled"

When you migrate to Itanium, some of the stuff you can do with the unsupported pi utility can be done with the Caliper tool. I'm reasonably certain though that Caliper only reports metrics from the Itanium2 PMU - Performance Management Unit and not from other parts of the CEC (Core Electronics Chipset - aka the "glue") Again, on simpler systems, the CPU memory bus loading _is_ the memory bus loading (rx[1245]XXX systems), and it will be the local memory bus loading for the cell-based systems (rx[78]XXX and 'dome) and again, best to have a system block diagram handy :)

[Bill Hassell]

Knowing the memory bus throughput (which is many Gb/sec) won't tell you much. An idle machine is constantly accessing memory in order to run the instructions. You can't infer that a slow application is because the memory bus is slow. Now Oracle would have to be configured quite uniquely to keep the database in RAM (SGA or shared memory is several Gb). You could serially read the datafiles to load up the buffer cache, but the real cause for delays is very likely Oracle configs and poor SQL code.

If your DBA has left out important indexes thinking that serial reads will be just as fast as indexed reads because every is in memory, suggest an Oracle performance class for the DBA to take. If sar reports that there are truly no signifcant disk accesses, then troubleshoot the delays by creating a reprodicbile query or transaction, then use Glance to look at the memory stats (top, vmstat, sar are useless for this task). If you see a high page fault rate (refers to the TLB hardware cache), you can enable large page size for the Oracle engine so that large amounts of memory can be accessed without a TLB miss. Show your DBA/programmers this page:

http://docs.hp.com/en/B2355-90655/ch08s02.html

[rick jones]

The quantity of bus traffic from a CPU in the idle loop should be very low. If not, it signifies a bug in the idle loop. Still, it would be interesting to know what Victor plans on doing with the data.