Problem while creating a process

[Prashant Zanwar]

Hi,

I am getting below error while I am trying to start a process in nohup. Even if I try to telnet to other server I get same error. Or even when I do a rcp from same server or from somewhere else, i get below error. Please let me know urgent if someone know solution. There is nothing abnormal on system. CPU is heavily used at present. And I am doubting on shared memory.

$ telnet crmu10
/usr/lib/dld.sl: Call to mmap() failed - TEXT /usr/lib/libsis.sl
/usr/lib/dld.sl: Not enough space
Abort(coredump)

Please reply, i will summerize...

[Kent Ostby]

Not enough space is generally out of swap on the box although it could be a system table.

ENOSPC 28 /* No space left on device */

[ENOSPC] No space left on device. During a write() to an
ordinary file, there is no free space left on the
device; or no space in system table during msgget(),
semget(), or semop() while SEM_UNDO flag is set.

check swapinfo -t on the box.

[RAC_1]

You are running out of memory. Check swapinfo -mat.

Also check your kernel parameters.

Check vmstat 5 5. Check for free mem. Also check glance -m and for free memory.

What your buffer cache settings. dbc_max_pct and dbc_min_pct?

[Kent Ostby]

If you are only seeing this with telnet, then it could be a problem with device files that has been seen from time to time when certain device files are removed:

To troubleshoot which files have been removed and must be recreated:

1) cd /dev/pty
2) ls|sed -e 's/tty//' | sort > file1
3) cd ../ptym
4) ls|sed -e 's/tty//' | sort > file2
5) diff file1 file2

Best regards,

Kent M. Ostby

[Prashant Zanwar]

root@ccau40:>kmtune -q dbc_min_pct
Parameter Value
===============================================================================
dbc_min_pct 5

root@ccau40:>kmtune -q dbc_max_pct
Parameter Value
===============================================================================
dbc_max_pct 10


Hi above is the setting of kernel parameters..

Swapinfo is showing something like below, i believe which is normal..

root@ccau40:>swapinfo -mt
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 1024 0 1024 0% 0 - 1 /dev/vg00/lvol_swap
dev 1024 0 1024 0% 0 - 1 /dev/vg00/lvol_swap2
reserve - 1896 -1896
memory 3131 1825 1306 58%
total 5179 3721 1458 72% - 0 -
I dont see anything abnormal, except these kernel parameters if these are!!

Please suggest...

[RAC_1]

what are the results of vmstat 5 5 and glance -m. Check for free mem.

[yogesh_4]

Hi,
please check #file core , #what core and #strings core output. If #file core gives SEGSIGV then it is segment violation problem.This problem is definatly replated to kernal parameters which are replated to memory. For SEGSIGV you can tune maxxsiz, maxtsiz parameters.

[Bill Hassell]

You'll need to run the swapinfo -tm command when the error is occuring. If the CPU is heavily used, then you may momentarily out of swap space so use swapinfo to 'borrow' some space from a filesystem with lots of space. The space will be returned upon reboot. Check also that maxdsiz is a reasonable value (at least 64megs).

[Prashant Zanwar]

vmstat is showing enough memory available.

And also swapinfo is alright.

I am attaching the file for details..

Please see and suggest..

[RAC_1]

Post the results of file core, what core and strings core commands.

Also do tusc -vfp "your command" and post.

Kernel parameters seem OK.

[Prashant Zanwar]

Please see the core file details..

This says cannot allocate memory..

Please suggest...

[Prashant Zanwar]

I cannot see tusc in path...
Can u tell where is that located...

Regards
Prashant

[Kent Ostby]

Are you seeing this with anything OTHER then telnet ?

Are you seeing it consistently, intermittantly, or just one time ?


If only telnet, did you check the device files as I listed above in response 2 ?

[Prashant Zanwar]

This is continuous problem...

Regards
-Prashant

[Kent Ostby]

The out of memory messages are related to the "strings" run .. thats not what is in the "core" file.

I would suggest looking to see if you have any processes that might be in a runaway state using lots of CPU ...

I'd suggest you do:

ps -ef | cut -c42-80 | sort -nr | head -15

Wait about two minutes and do it again and see which processes are gaining in CPU usage.

If you spot one that is using a lot and shouldnt be then killing that process off would probably free up whatever resource is having a problem.

[Prashant Zanwar]

2575:23 oracleWPSRP (LOCAL=NO)
607:48 statdaemon
326:23 /usr/sbin/syncer
167:07 vxfsd
88:10 ora_snp0_WPSDV
61:45 psmctd
52:11 cclogd
51:55 /opt/perf/bin/alarmgen -svr 3389
45:54 /usr/sbin/pwgrd
35:04 bgscollect -I noInstance -M /opt
31:18 swapper
27:32 opcmona
27:02 vhand
25:39 ora_snp0_WPSTR
24:45 /opt/perf/bin/rep_server -t SCOP

Below is output of above mentioned...

Please suggest

[Prashant Zanwar]

Hi,

I dont know how, but this problem is no more existing. It got resolved on it own.

Can someone plese tell how to see shared memory usage and what is maximum allowed for a process and how a process is using shared memory?

Thanks to all who replied...and will reply..