RAM memory occupied by operating system

[singaravelan]

Hi All,
Basically,I am a Oracle Database Administrator.
We manage oracle databases on HP-SUPERDOME servers.Version PA-RISC 11.11.

Our doubt is in the RAM memory allocation on the server.
When we type glance and press M it displays the memory allocation.
For our system it is as below
Total VM : 12.2gb
Active VM: 2.09gb
-----------------------------------------------
Phys Mem: 24.0gb
Sys Mem : 8.61gb
User Mem: 9.22gb
Buf Cache: 490.8mb
Free Mem: 5.65gb

Now (sys + user + buf cache + free) = TOTAL RAM

We need to know why Sys Mem is so high on our server.
We have many servers of the same type and configuration but still the sys memory is varying from system to sytem.

Since the database that runs on this server supports about 4000 concurrent sessions we face memory crunch.
Does it have anything to do with the kernel parameters?
I am attaching the output of KMTUNE.

[Rasheed Tamton]

Hi Singaravelan,

Can you give us the output of

swapinfo -tam
vmstat 4 10
sar -b 4 10

[singaravelan]

Hi,

Thanks for your interest in solving our doubt.
I have attached the required outputs.

Thanks and Regards,
Singaravelan.

[Eric Antunes]

Hi,

Any reason for swapmem_on=0? Did you change it or is it the default for superdome (kmtune -lq swapmem_on)?

Do you really need ninode=40000? Please post the output of "sar -v 3 33".

Best Regards,

Eric Antunes

[singaravelan]

Hi Eric,

Thanks for the reply.
I have attached the required output.

Basically I am an Oracle DBA.I do not manage the Unix servers.If you want to diagnose something or want specific answers for something you let me know.I could get the answers from my Unix admins.

Thanks and Regards,
Singaravelan.

[Eric Antunes]

Hi Singaravelan,

As you can see from the sar output, you don't need that ninode size. So I recomend you to decrease (or ask your system adm. to decrease) this parameter value starting with 2048 and monitoring it.

You didn't post the output of "kmtune -lq swapmem_on"...

Best Regards,

Eric Antunes

[Eric Antunes]

Hi again,

About ninode, you can also keep it to the default value: as root, go to sam -> Kernel Configuration -> [select ninode] Actions -> Mofify Configurable Parameters -> Use Formula in Source File. This will use the default formula: ((NPROC+16+MAXUSERS)+32+(2*NPTY))

Best Regards,

Eric Antunes

[singaravelan]

Hi Eric,

Thanks for the updates.
I will discuss about your updates with our system administrators.

I have attached the required output below

kmtune -lq swapmem_on
Parameter: swapmem_on
Current: 0
Planned: 0
Default: 1
Minimum: -
Module: -
Version: -
Dynamic: No

Thanks and Regards,
Singaravelan.

[singaravelan]

Hi Eric,

Just a small clarification.

On a similar system with 12 CPUs and 48 GB RAM the system memory usage is only 5GB.
There the ninode value is set to 160000.

On the problematic system there are 12 CPUs and 24 GB RAM but the system memory usage is 9GB with ninode value of 40000.

Will anyother parameter other than ninode also affect the total memory used by system.

Thanks and Regards,
Singaravelan.

[Steve Lewis]

Yes, another memory hog that is commonly over-allocated is semaphores.

Check your kernel parameters
semmni, semmnu, semmns, semume

Some of these such as semume can cause huge memory allocations.
See how many you are actually using by checking ipcs -sa and also checking within Oracle for latch usage.

With 2000 concurrent users you will need a reasonable number.

Some vx kernel parameters can also cause large allocations of memory due to unlimited memory being the default.

[Rasheed Tamton]

Hi,

Your vmstat shows the scan rate bit high (sr 13)

Can you compare the below values with the other system - which you say quite ok.
vx_ninode
nbuf
bufpages
dbc_min_pct
dbc_max_pct
swapmem_on

Does both the systems use raw devices for oracle or file systems. If file systems just check the mount options

For shared mem info:
ipcs -bmop

UNIX95= ps -e -o ruser,vsz,pid,args | sort -rnk2 | more
This will show processes using memory from the max to min
(hope Bill Hassell might come in to give more explanation)

[Eric Antunes]

Hi Singaravelan,

ninode may not be your issue but it is always good idea to limit it to reasonable values for you workload.

swapmem_on should only be 0 when you are SURE you have enough swap disk to cover your expected workload.

Please excute the attached script (as root and you must input a starting memory size in Mb) and post the output. For example:

./show_heavy_processes.sh 100

Best Regards,

Eric Antunes

[singaravelan]

Hi All,
Thank you all for the keen interest shown towards solving our issue.

Hi Rashid,
Oracle 9i database is running on that server and it uses file system.

Hi Steve,
We will compare the semaphore parameters on both the systems.

Hi Eric,
I have asked the unix admins to run the script.It may take a while to get the output.

In the meanwhile I have attached the kmtune output of both the systems.If possible please review to see if anything has been set wrongly or too high a value.

Also the dbc_min_pct is set to 1 and dbc_max_pct is set to 2 because we run Oracle database and it has its own cache.

The only difference we observe is that the system memory is 9GB for one and 4GB for the other.

Thanks and Regards,
Singaravelan.

[singaravelan]

Hi,

The previous attachment is for the system with less system memory.

This one is for the system with more system memory.

Thanks and Regards,
Singaravelan.

[Eric Antunes]

Hi ,

You can execute it in the command line as root:

UNIX95= ps -e -o vsz= -o pid= -o ppid= -o time= -o state= -o flags= -o comm|sort -rnk2|more

Pay special attention to java processes if you use java...

Best Regards,

Eric Antunes

[Rasheed Tamton]

Hi Singaravelan,

just a quick thought. I did not check the attachments yet. But your other system is having 48GB RAM and I prusume at least it has 48GB swap space also. So that makes some difference there if you run similiar oracle setup there.

HP recommendation for oracle for swap space is to have twice the size physical memory. If you have a large memory, it may not be necessary to create swap space thatâ s twice the size of physical memory but since disk storage is now cheap compared to memory, the suggestion is to create large swap space.

Regards,
Rasheed Tamton.

[singaravelan]

Hi,

I have attached the output of the UNIX95 command.

Hi Rasheed,

We have our SWAP space equal to the physical RAM.

Thanks and Regards,
Singaravelan.