Physical memory becoming low

[Danesh Qureshi]

Hello,

I am currently running the following operating system on HP Proliant DL380 server:-
Red Hat Enterprise Linux AS 4 (Nahant Update 3).

The HP Proliant DL380 server has 3Gb of main memory in it.
I have noticed the memory is becoming very low.

When I do a vmstat -s the total memory is 3115272K and the used memory is 3096768K. The server has an Oracle database
which has been allocated 300Mb of memory for expansion which is very low.

Can anyone tell me what else I can do to find out what is causing the memory to become low and how I can fix
the low memory problem?
Also top is also displaying the main memory is getting low. See output of top below:-

$ top
top - 15:47:59 up 60 days, 1:18, 2 users, load average: 0.00, 0.00, 0.00
Tasks: 86 total, 1 running, 85 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0% us, 0.0% sy, 0.0% ni, 100.0% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 3115272k total, 3096656k used, 18616k free, 84792k buffers
Swap: 4194288k total, 63416k used, 4130872k free, 2817452k cached

[Ivan Ferreira]

Check this thread:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1029490

[Steven E. Protter]

Shalom,

Oracle is relatively famous for memory leaks and other issues. It could be after running for 60 days time to bounce the system or the database.

It is possible your system is working harder and consuming more memory. Or it could be you need an Oracle patch.

SEP

[Stuart Browne]

Yeah, that thread Vitaly posted is a good start.

As is stated there, Linux will try to use all available memory.

As you can see from your 'top' output there, there's 2.8GB used in the 'cached' value. These are dynamically allocated disk-cache using available physical memory to speed up disk access.

As the requirement for memory increases, this value is usually the first to go down, followed by the 84MB of 'buffers'.

With regards to the utilisation of physical memory, what you want to watch is the Swap 'used' value. If this increases steadily, then you're starting to be in trouble.

You mentioned Oracle, and SEP mentioned memory leaks. This certinaly might be possible (there certainly are enough in all applications!). You can prove or disprove this by restarting Oracle in a quite time. If the Mem 'free' and 'cached' values jump up significantly whilst Swap 'used' drops and they don't grow (quickly) to similar values after restarting it, then you have a memory leak in progress (bear in mind Oracle like all databases take a while to get to their optimum memory usage values for query caches etc. etc.).

If it is a memory leak, check for version updates you can do to fix the issues, or take the slackers way out of scheduling a restart periodically (just for the record, I'm one of those slackers. 'saslauthd' with 'pam_mysql' are the bane's of my existance! There are only so many memory leaks you can hunt down before frustration sets in!).

Looking at those values though, I'd say things are running well. You've still got plenty of 'cached' value, so you're not exhausting the system yet.

[Danesh Qureshi]

I have attached a document which uses the free command on this system to display the behaviour of the cache memory ,etc.

Please can you tell me what is going on here with the memory values before and after shutting down the Oracle database?

Thank you.

[Ivan Ferreira]

From the first free output, you can see that there are 2732 m cached. That means that most memory is used as buffers/cache. The database memory usage is very low, you should check your shared memory kernel parameters and database parameters.

After shutting down the database, you still have 2636 m used and your memory used by apps drops down to 125 m.

Can you post the output of ipcs -a when the database is up?

[George Liu_4]

From the output of "top". The memory status is good, actually. Cached potion could be freed when never memory demands rise

[Danesh Qureshi]

Here is the output from ipcs for you to examine:-

[root@metis sysconfig]# ipcs -a

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x3a6aeae8 458752 oracle 640 312475648 18

------ Semaphore Arrays --------
key semid owner perms nsems
0xa1c77978 1933312 oracle 640 154

------ Message Queues --------
key msqid owner perms used-bytes messages

[root@metis sysconfig]#

[Bill Thorsteinson]

I would expect almost all physical memory to
be used. O/S is designed to cache as much
disk data as possible. Once you've read 3GB
of disk data memory should be full.
Significant use of swap space tends to indicate
that you have a pending problem with memory
utilization.

Check memory usage shortly after startup.
If you have lots of physical memory availalbe,
then you should not have a problem.

Oracle tends to map a huge share memory area,
the SGA to all processes. I have had
problems with both system admins and DBAs
complaining about the size of Oracle
processes. Check the memory footprint of
all your Oracle processes. They should
be about the same large size. Memory leaks
will show up as a single or couple of
processes significantly larger than the rest
and growing in size.

I haven't run into memory leaks in Oracle
for quite a while.
My last memory leak problem was with the
monitor that was watching for memory leaks.
I watched the system slowly die as the
monitor swallowed up all the swap space.

[Ivan Ferreira]

From the output of ipcs -a it's possible to view that only 312475648 bytes (312 MB) is configured to be used by oracle. You should increase this size for example, to 2 GB if oracle is the primary service in this server.

[Stuart Browne]

Wow, I can't beleive I mussed up that first sentance of my last post. Sorry Ivan!

[Danesh Qureshi]

I would like to point out there have been no performance issues on this system. I thought you may be able to shed some light on the output of the free command. See below:-

[root@metis sysconfig]#
# free -m
total used free shared buffers cached
Mem: 3042 3020 21 0 153 2670

-/+ buffers/cache:
196 2845
Swap:
4095 47 4048

So looking at these figures the cached memory is high which indicates the system can make use of cached memory even though main memory is running low. Also there appears to be some swapping taking place but not affecting system performance.

In conclusion the memory utilization values are indicating the system appears to be healthy and so there is no cause for concern as long as there plenty of cached memory available to the system. Hence, am I right in saying this?