HPE Community read-only access December 15, 2018
This is a maintenance upgrade. You will be able to read articles and posts, but not post or reply.
Hours:
Dec 15, 4:00 am to 10:00 am UTC
Dec 14, 10:00 pm CST to Dec 15, 4:00 am CST
Dec 14, 8:00 pm PST to Dec 15, 2:00 am PST
cancel
Showing results for 
Search instead for 
Did you mean: 

Mem 100% full

 
Pradep
Regular Advisor

Mem 100% full

Greetings,
Could you please help me find the issue with hpux11iv3 over PA-RISC. / OracleR12

B3692A Glance C.04.70.000 15:09:22 uasng076 9000/800 Current Avg High
-------------------------------------------------------------------------------------
CPU Util SSUU | 7% 5% 7%
Disk Util FV V | 6% 5% 9%
Mem Util S SU| 99% 99% 99%
Swap Util U UR R | 37% 37% 37%
-------------------------------------------------------------------------------------
MEMORY REPORT Users= 5
Event Current Cumulative Current Rate Cum Rate High Rate
--------------------------------------------------------------------------------
Page Faults 6096 12798 1172.3 688.0 1172.3
Page In 214 530 41.1 28.4 328.2
Page Out 0 0 0.0 0.0 0.0
KB Paged In 856kb 2.1mb 164.6 113.9 852.0
KB Paged Out 0kb 0kb 0.0 0.0 0.0
Reactivations 0 0 0.0 0.0 0.0
Deactivations 0 0 0.0 0.0 0.0
KB Deactivated 0kb 0kb 0.0 0.0 0.0
VM Reads 226 576 43.4 30.9 52.3
VM Writes 0 0 0.0 0.0 0.0
Total VM : 4.4gb Sys Mem : 15.9tb User Mem: 70.3gb Phys Mem : 8.0gb
Active VM: 1.8gb Buf Cache: 6mb Free Mem: 65mb FileCache: 2.2gb
MemFS Blk Cnt: 0 MemFS Swp Cnt: 0


SWAP SPACE Users= 5

Swap Device Type Avail Used Priority
--------------------------------------------------------------------------------
/dev/vg00/lvol2 device 16.0gb 709mb 1
/dev/vg00/lvol10 device 1.1gb 134mb 1
pseudo-swap memory 7.6gb 5.4gb -1



# swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 16384 709 15675 4% 0 - 1 /dev/vg00/lvol2
dev 1136 134 1002 12% 0 - 1 /dev/vg00/lvol10
reserve - 2949 -2949
memory 7790 5495 2295 71%
total 25310 9287 16023 37% - 0 -

13 REPLIES
Pradep
Regular Advisor

Re: Mem 100% full

Oops. Oracle DB version is 11g.

Re: Mem 100% full

>Mem 100% full

This is a good thing, what's the problem? Do you want to add more load on the system?

>Page Out 0 0 0.0 0.0 0.0

No page outs.

>memory 7790 5495 2295 71%

At this time, you have 2 Gb free and lots of device swap.
Pradep
Regular Advisor

Re: Mem 100% full

we had a issue reported by DBA that commands were kind of hung. then we found that Glance is showing 100% Mem.
can you please help me understand why glance shows mem 100%, even though Psuedo-swap which is part of physical memory is 25% free ? what defines the size of pseudo-swap.any kernel parameter ?

how do i interpret Total VM : 4.4gb Sys Mem : 15.9tb User Mem: 70.3gb. Mem is 15.9TB ? and user Mem 70gb ?

Should i decrease size of FileCache: from 2.2gb to 1GB..to create extra available space in Memory ?

Re: Mem 100% full

>we had a issue reported by DBA that commands were kind of hung. then we found that Glance is showing 100% Mem.

Did you collect this info when it was still hung?

>what defines the size of pseudo-swap, any kernel parameter?

The size of RAM times a percent. Search for Don Morris' comments on it.

>Should I decrease size of FileCache: from 2.2gb to 1GB to create extra available space in Memory?

That would be helpful if you need it for your SGA.
Pradep
Regular Advisor

Re: Mem 100% full

Thanks Dennis.
Can you tell me why glance shows mem 100%.
Whats the meaning of Sys Mem : 15.9tb User Mem: 70.3gb
They are far above total swap on server.

Re: Mem 100% full

>Can you tell me why glance shows mem 100%.

It may be related to your next inconsistency?

>What's the meaning of Sys Mem: 15.9tb User Mem: 70.3gb? They are far above total swap on server.

Right they do seem incorrect. Have you contacted the Response Center about the problem? Do you have the latest Glance?
chris huys_4
Honored Contributor

Re: Mem 100% full

Hi,

> Mb Mb Mb PCT START/ Mb
> TYPE AVAIL USED FREE USED LIMIT RESERVE PRI
> NAME
> dev 16384 709 15675 4% 0 - 1 /dev/vg00/lvol2
> dev 1136 134 1002 12% 0 - 1 /dev/vg00/lvol10
The "larger then 0% USED percentage output" for the device swapspace, could be an indication that at one time, the system was actually "using" device swap space as "extra memory", which would result in a not very responsive system..

Check the SGA size of the different databases on the system and see if they are not to "close" to using all of the 8gb of physical memory in the system.

Run also a "kmeminfo;kmeminfo -user;crashinfo -continue" as a root cronjob every 5 minutes and monitor the memory usage.

Greetz,
Chris
B. Hulst
Trusted Contributor

Re: Mem 100% full

How much memory is allocated to Oracle?
What does it tell at the startup in the alert_.log?

You could have too little SGA space or wrong setup of the shared pool,etc memory configuration.

And what is the max.number of Oracle connections possible and how many do you have on average?
V.P
Frequent Advisor

Re: Mem 100% full

Hi Pradep,

Please check whether the swapmemory is on or not?

kctune|grep -i swapmem_on

This should be on (1)
VK2COT
Honored Contributor

Re: Mem 100% full

Hello,

a) Kernel parameter swapmem_on does not
exist on HP-UX 11.31. Pseudoswap is
enabled by default.

b) Having two paging devices on the same
physical disk(s) is a very bad idea. You
have two of them in volume group vg00 and
most probably they are on the same disk(s).

Please review it and remove one of them.

Your design of paging devices breaks almost
all best practices:

a) One swap device per physical disk;
b) Swap devices should be of preferably of
same size;
c) Spread swap devices across fast disks;
and so on.

c) In regards to 100% RAM utilisation,
others already gave you some good pointers,
starting with Unified File Cache tuning.

Cheers,

VK2COT
VK2COT - Dusan Baljevic
Pradep
Regular Advisor

Re: Mem 100% full

Thanks to all.
I had logged case with HP Response center.
following Params were tuned to resolve it.
because kmeminfo showed like this.

Physical memory = 2096640 8.0g 100%
Free memory = 40007 156.3m 2%
User processes = 785502 3.0g 37% details with -user
Detached SHMEM = 1 4.0k 0% details with -shmem
System = 1268660 4.8g 61%
Kernel = 732732 2.8g 35% kernel text and data
File Cache = 535928 2.0g 26% details with -ufc

#kctune filecache_max=8%
#kctune filecache_min=5%
#kctune vxfs_ifree_timelag=-1
#kctune vx_ninode=40000

SGA is 1GB.
SoorajCleris
Honored Contributor

Re: Mem 100% full

hmmm

in 11.31 vx_ninode is always suspected when there is a memory issue.

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&objectID=c01527093

Regards,
Sooraj
"UNIX is basically a simple operating system, but you have to be a genius to understand the simplicity" - Dennis Ritchie
Pradep
Regular Advisor

Re: Mem 100% full

reply above