couldn't fork child process: not enough memory

[Brian Bientz]

We use Oracle Enterprise Manager to monitor our database servers. Periodically on our development server the OEM Intelligent Agent is reporting the following in its trace file:

diskstats: open failed: couldn't fork child process: not enough memory

The server has either 6 or 8 GB of which only about half is allocated to Oracle instances. What does this mean? Is there a kernal parameter that needs to be changed?

[RAC_1]

When this error is reported, check following.

swapinfo -mat
glance -m
vmstat (check free column)

Anil

[Ted Buis]

Could you provide the list of kernel parameter values too?

[A. Clay Stephenson]

Always, always when you see errors, consult the section 2 man pages.

Do a man 2 fork and you will see when it sets errno = ENOMEM. You are almost certainly running out of virtual memory -- you need more swap space or better you need to reduce the load --- smaller SGA's. Do a vmstat and look at the pageout rate; if it's significant then you need to add more memory or reduce the SGA's. Until you can get more memory (even if you add more swap) you really need to reduce the SGA's because the overhead of having to go to the disks more often for database i/o is trivial (by a factor of 100) to the overhead of having to page.

[Kent Ostby]

This is generally a lack of swap.

Check swapinfo and make sure that your box is configured with at least 2xRAM worth of disk swap (the general recommendation from HP).

Best regards,

Oz

[Ted Buis]

Specifically, I wanted to check to see what you had set for the max value for dynamic buffer cache in your kernel.