Unable to stop sendmail

[NDO]

Hi

I am having an issue with a server running HP-UX 11.31, and for some reason /var/spool/mqueue has almost seven hundred thousand files, and I am trying to clear them using the following script:

mailq | awk 'substr($0,1,1) !~ "[\t -]" { print $1 }' | while read i; do
rm /var/spool/mqueue/?f$i
done

but the performance is being impacted, and when I run :

UNIX95=1 ps -e -o vsz,pid,ppid,args | sort -rn | head -20

I got the following output:

617376  5959  5956 /opt/wbem/lbin/cimprovagt 0 5 10 root SFMProviderModule
  66688 10679     1 /opt/java6/bin/IA64N/java -Djava.util.logging.config.file=/u01/easydone/EASYDONE/GUI/Tomcat6_JOSSO_8022/conf/logging.properties
  47904 14307 12697 sendmail: ./tAK1JAX6014873 from queue
  47904 13639 12697 sendmail: ./tAJNWZZN014173 from queue
  47904 13000 12697 sendmail: ./tAK1JAWE014873 from queue
  47904 12344 12697 sendmail: ./tAK1JASI014873 from queue
  47904 11717 12697 sendmail: ./tAJNWZiX014173 from queue
  47904 11039 12697 sendmail: ./tAJH4Q38013456 from queue
  47888 17510 12697 sendmail: ./tAJFGQCD015559 from queue
  47888 16864 12697 sendmail: ./tAJNWZjX014173 from queue
  47888 16201 12697 sendmail: ./tAJNWZYh014173 from queue
  47888 15571 12697 sendmail: ./tAJH4Q1w013456 from queue
  47888 14903 12697 sendmail: ./tAK1JAdE014873 from queue
  47872 21511 12697 sendmail: ./tAJIe7hU012698 from queue
  47872 20877 12697 sendmail: ./tAKBBiKp015559 from queue
  47872 20227 12697 sendmail: ./tAK1JAd5014873 from queue
  47872 19611 12697 sendmail: ./tAJIe7f9012698 from queue
  47872 18819 12697 sendmail: ./tAJFGQ9s015559 from queue
  47856 27506 12697 sendmail: ./tAK1JAUm014873 from queue
  47856 26799 12697 sendmail: ./tAJNWZic014173 from queue

So when I try to stop sendmail, I got the folloing:

/sbin/init.d/sendmail stop
Unable to shut down sendmail.....

Please can someone give me a hint on what could be wrong?

[Patrick Wallek]

I would probably kill all the sendmail processes manually and then restart the daemon.

[Dennis Handly]

You could look at my script here for how to remove only specific messages:

http://community.hpe.com/t5/Messaging/Help-in-clearing-out-var-spool-mqueue/m-p/5276091#M5446

>can someone give me a hint on what could be wrong?

As Patrick said, you could just clean them up manually.  tusc(1) might be able to tell you what these sendmail child processes are doing.

[NDO]

Hi

I believe I found the reason for all this slowness, I read the contents of some of message files in /var/spool/mqueue, like and there were error messages regarding files that do not exist, issues with oracle not exist:

dftB1IA1PE02065

EXP-00056: ORACLE error 1034 encountered
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
HPUX-ia64 Error: 2: No such file or directory
EXP-00005: all allowable logon attempts failed
EXP-00000: Export terminated unsuccessfully
mkdir: cannot create /bd/DUMPS/2015-12-01: File exists
/u01/easydone/EASYDONE/GUI/DB_DMP_LIB/edimo_dmp.sh[19]: bscs_tables.sql: cannot open

Export: Release 11.2.0.3.0 - Production on Tue Dec 1 22:00:07 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.


EXP-00056: ORACLE error 1034 encountered
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
HPUX-ia64 Error: 2: No such file or directory
EXP-00005: all allowable logon attempts failed
EXP-00000: Export terminated unsuccessfully
mkdir: cannot create /bd/DUMPS/2015-12-01: File exists

Export: Release 11.2.0.3.0 - Production on Tue Dec 1 22:00:07 2015

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.


EXP-00056: ORACLE error 1034 encountered
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
HPUX-ia64 Error: 2: No such file or directory
EXP-00005: all allowable logon attempts failed
EXP-00000: Export terminated unsuccessfully
/u01/easydone/EASYDONE/GUI/DB_DMP_LIB/sfeco_dmp.sh[21]: gzip:  not found


*************************************************
Cron: The previous message is the standard output
      and standard error of one of your crontab commands:

/u01/easydone/EASYDONE/GUI/DB_DMP_LIB/dmp_easydone.sh > /u01/easydone/EASYDONE/GUI/logs/crone.easydone.dmp.log

[Patrick Wallek]

You really need to take a look at your systems health.

The problem is not that files don't exist.  The problem is that you have run out of shared memory.  This is causing your script to not be able to login to Oracle. 

If this script isn't running, what else isn't running?  You have bigger problems going on than just a sendmail issue.

[NDO]

Hi

Yes I have address this oracle issues to the DBA, as I am not a DBA, is going to take care of it. I will probably need to change some kernel parameters, I am not sure, but I am investigating. But the application now is running smoothly