CLOSE_WAIT HP-UX 11.31

NDO · ‎12-10-2014

Hi!

I have a number of servers, that suddenly not sending mails, or alerts using SNMP, I have run the following commands such as:

date  |  mailx  -s  "Local sendmail Test" <my email address>

and I did not get nothing, them:

netstat -anf inet | grep 161
tcp        0      0  127.0.0.1.7161         *.*                     LISTEN
tcp        0      0  10.100.48.14.62161     10.1.20.238.30300       CLOSE_WAIT
udp        0      0  *.161                  *.*

decided to start/stop snmp:

/sbin/init.d #./SnmpMaster stop
snmpdm stopped

./SnmpMaster start
SNMP Research SNMP Agent Resident Module Version 15.3.1.0
Copyright 1989, 1990, 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001 SNMP Research, Inc.

and run again netstat:

#netstat -anf inet | grep 161
tcp        0      0  10.100.48.14.62161     10.1.20.238.30300       CLOSE_WAIT
tcp        0      0  10.100.48.14.63161     10.1.20.238.63992       ESTABLISHED
tcp        0      0  127.0.0.1.7161         *.*                     LISTEN
udp        0      0  *.161                  *.*

and I still see:

tcp        0      0  10.100.48.14.62161     10.1.20.238.30300       CLOSE_WAIT

Please can somoeone explay the CLOSE_WAIT status, is this somethin to do with the issue I am facing?

Eric Antunes · ‎12-11-2014

Hi,

Do you already have the following patches installed?

PHNE_39709 - freezing sockets, cumulative ARPA Transport patch
PHNE_40428 - freezing sockets, STREAMS Cumulative Patch

If not, try them.

Best Regards,

Eric

Each and every day is a good day to learn.

NDO · ‎12-11-2014

Hi

I dont have:

#swlist | grep -i PHNE_39709
mcel-dunn02[141]/ #swlist | grep PHNE_40428
mcel-dunn02[142]/ #

But the snmp was working last week, the patch installation unfortunately I cannot do it, because os type of contract we have with HP, that does not allows us to download patchs

NDO · ‎12-11-2014

The above patchs are also not present on a system that is sending me alerts, and emails, so I beleive the patch installation will not change anything

Eric Antunes · ‎12-11-2014

And those other systems are HP-UX 11.31?

Each and every day is a good day to learn.

Patrick Wallek · ‎12-11-2014

The e-mail issue is likely to be different from the SNMP issue.

With regards to the e-mails -- Do you see any errors in the /var/adm/syslog/mail.log file?

Also, what does the 'mailq' command show?

NDO · ‎12-11-2014

All systems are running 11.31, and in the logs I´ve got:

date | mailx -s "test" fretagi@mcel.co.mz
mcel-dunn02[167]/ #tail  /var/adm/syslog/mail.log
Jul 10 00:18:03 mcel-dunn02 sm-mta[9129]: s69GJ7TG009129: to=root, delay=00:00:03, mailer=local, pri=38825, stat=queued
Jul 10 00:18:05 mcel-dunn02 sm-mta[5764]: s613RbEn021081: to=root, delay=8+17:29:52, mailer=local, pri=127318, dsn=4.4.7, stat=Cannot send message for 5 days
Jul 10 00:18:05 mcel-dunn02 sm-mta[5764]: s613RbEn021081: s69GTDdg005764: return to sender: Cannot send message for 5 days
Jul 10 00:18:09 mcel-dunn02 sm-mta[5764]: s69GTDdg005764: to=root, delay=00:00:04, mailer=local, pri=38825, stat=queued
Jul 10 00:18:14 mcel-dunn02 sm-mta[23125]: grew WorkList for /var/spool/mqueue to 365000
Jul 10 00:18:18 mcel-dunn02 sm-mta[9129]: s2TM8s7U013488: to=root, delay=101+23:52:00, mailer=local, pri=127318, dsn=4.4.7, stat=Cannot send message for 5 days
Jul 10 00:18:18 mcel-dunn02 sm-mta[9129]: s2TM8s7U013488: s69GJ7TH009129: return to sender: Cannot send message for 5 days
Jul 10 00:18:19 mcel-dunn02 sm-mta[5764]: s613RbBi021081: to=root, delay=8+17:40:10, mailer=local, pri=127318, dsn=4.4.7, stat=Cannot send message for 5 days
Jul 10 00:18:19 mcel-dunn02 sm-mta[5764]: s613RbBi021081: s69GTDdh005764: return to sender: Cannot send message for 5 days
Jul 10 00:18:20 mcel-dunn02 sm-mta[5764]: s69GTDdh005764: to=root, delay=00:00:01, mailer=local, pri=38825, stat=queued
Jul 10 00:18:22 mcel-dunn02 sm-mta[9129]: s69GJ7TH009129: to=root, delay=00:00:04, mailer=local, pri=3882mcel-dunn02[168]/ #
mcel-dunn02[168]/ #

and the mail transporter is pinging...

Eric Antunes · ‎12-11-2014

Check if the other systems don't have patches that supersedes PHNE_39709 and PHNE_40428 (In my HP-UX 11.31 systems, for example, I have ARPA PHNE_41714 and STREAMS PHNE_41060 patches).

Please post the output of for the different servers:

swlist -l patch |grep ARPA; swlist -l patch |grep STREAMS

Each and every day is a good day to learn.

NDO · ‎12-11-2014

#swlist -l patch |grep ARPA; swlist -l patch |grep STREAMS
# PHCO_38007                            1.0            cumulative ARPA Transport Japanese manpage patch
# PHNE_38680                            1.0            cumulative ARPA Transport patch
# PHNE_37257                            1.0            STREAMS Cumulative Patch
# PHNE_37257.STREAMS-64SLIB                1.0               Streams.STREAMS-64SLIB    applied
# PHNE_37257.STREAMS-KRN                   1.0               Streams.STREAMS-KRN    applied
# PHNE_37257.STREAMS-MIN                   1.0               Streams.STREAMS-MIN    applied
# PHNE_37257.STREAMS2-KRN                  1.0               Streams.STREAMS2-KRN    applied
# Streams.STREAMS-64SLIB                   B.11.31           Streams 64 bit Shared User Libraries
  PHNE_37257.STREAMS-64SLIB        1.0               Streams.STREAMS-64SLIB    applied
# Streams.STREAMS-KRN                      B.11.31           HP-UX Kernel support for network streams
  PHNE_37257.STREAMS-KRN           1.0               Streams.STREAMS-KRN    applied
# Streams.STREAMS-MIN                      B.11.31           Streams 32 bit Shared User Libraries
  PHNE_37257.STREAMS-MIN           1.0               Streams.STREAMS-MIN    applied
# Streams.STREAMS-RUN                      B.11.31           Streams include files and catalog files
# Streams.STREAMS2-KRN                     B.11.31           HP-UX Kernel Network Streams library
  PHNE_37257.STREAMS2-KRN          1.0               Streams.STREAMS2-KRN    applied
# Streams.STREAMS2-RUN                     B.11.31           Streams commands and utilities
prep01[185]/ #

The above is for a system with NO PROBLEMS.

for a system with the issue:

swlist -l patch |grep ARPA; swlist -l patch |grep STREAMS
# PHCO_40146                            1.0            cumulative ARPA Transport Japanese manpage patch
# PHNE_39709                            1.0            cumulative ARPA Transport patch
# PHNE_40428                            1.0            STREAMS Cumulative Patch
# PHNE_40428.STREAMS-64SLIB                1.0               Streams.STREAMS-64SLIB    applied
# PHNE_40428.STREAMS-KRN                   1.0               Streams.STREAMS-KRN    applied
# PHNE_40428.STREAMS-MIN                   1.0               Streams.STREAMS-MIN    applied
# PHNE_40428.STREAMS2-KRN                  1.0               Streams.STREAMS2-KRN    applied
# Streams.STREAMS-64SLIB                   B.11.31           Streams 64 bit Shared User Libraries
  PHNE_40428.STREAMS-64SLIB        1.0               Streams.STREAMS-64SLIB    applied
# Streams.STREAMS-KRN                      B.11.31           HP-UX Kernel support for network streams
  PHNE_40428.STREAMS-KRN           1.0               Streams.STREAMS-KRN    applied
# Streams.STREAMS-MIN                      B.11.31           Streams 32 bit Shared User Libraries
  PHNE_40428.STREAMS-MIN           1.0               Streams.STREAMS-MIN    applied
# Streams.STREAMS-RUN                      B.11.31           Streams include files and catalog files
# Streams.STREAMS2-KRN                     B.11.31           HP-UX Kernel Network Streams library
  PHNE_40428.STREAMS2-KRN          1.0               Streams.STREAMS2-KRN    applied
# Streams.STREAMS2-RUN                     B.11.31           Streams commands and utilities
itc01[343]/ #

for another system with the same issue:

swlist -l patch |grep ARPA; swlist -l patch |grep STREAMS
# PHCO_38007                            1.0            cumulative ARPA Transport Japanese manpage patch
# PHNE_38680                            1.0            cumulative ARPA Transport patch
# PHNE_37257                            1.0            STREAMS Cumulative Patch
# PHNE_37257.STREAMS-64SLIB                1.0               Streams.STREAMS-64SLIB    applied
# PHNE_37257.STREAMS-KRN                   1.0               Streams.STREAMS-KRN    applied
# PHNE_37257.STREAMS-MIN                   1.0               Streams.STREAMS-MIN    applied
# PHNE_37257.STREAMS2-KRN                  1.0               Streams.STREAMS2-KRN    applied
# Streams.STREAMS-64SLIB                   B.11.31           Streams 64 bit Shared User Libraries
  PHNE_37257.STREAMS-64SLIB        1.0               Streams.STREAMS-64SLIB    applied
# Streams.STREAMS-KRN                      B.11.31           HP-UX Kernel support for network streams
  PHNE_37257.STREAMS-KRN           1.0               Streams.STREAMS-KRN    applied
# Streams.STREAMS-MIN                      B.11.31           Streams 32 bit Shared User Libraries
  PHNE_37257.STREAMS-MIN           1.0               Streams.STREAMS-MIN    applied
# Streams.STREAMS-RUN                      B.11.31           Streams include files and catalog files
# Streams.STREAMS2-KRN                     B.11.31           HP-UX Kernel Network Streams library
  PHNE_37257.STREAMS2-KRN          1.0               Streams.STREAMS2-KRN    applied
# Streams.STREAMS2-RUN                     B.11.31           Streams commands and utilities
mcel-dunn02[172]/ #

Eric Antunes · ‎12-11-2014

As you can see the "good" server has ARPA and STREAMS cumulatives patches PHNE_38680 and PHNE_37257 but the "bad" server has ARPA and STREAMS cumulatives patches PHNE_39709 and PHNE_40428.

In your plave I would upgrade the bad server's patches PHNE_39709 and PHNE_40428 with the patches that supersedes them.

Each and every day is a good day to learn.

Eric Antunes · ‎12-11-2014

But the 2nd "bad" server has the same patches of the "good" one so it may not be a patch issue...

Each and every day is a good day to learn.

NDO · ‎12-12-2014

Could it be that the issue lies in :

Dec 12 10:50:54 itc01 sendmail[2231]: sBC8opfF002229: sBC8osfE002231: DSN: Host unknown (Name server: 192.168.1.39: host not found)
Dec 12 10:50:54 itc01 sendmail[2231]: sBC8osfE002231: to=root, delay=00:00:00, xdelay=00:00:00, mailer=local, pri=31530, dsn=2.0.0, stat=Sent

or could it be:

hosts:        dns files
ipnodes:      dns [NOTFOUND=return] nis [NOTFOUND=return] files
networks:     nis [NOTFOUND=return] files
protocols:    nis [NOTFOUND=return] files
rpc:          nis [NOTFOUND=return] files
publickey:    nis [NOTFOUND=return] files
netgroup:     nis [NOTFOUND=return] files
automount:    files nis
aliases:      files nis

in lines "hosts: dns" ??

Eric Antunes · ‎12-12-2014

Can you ping your DNS (192.168.1.39?) from all servers?

Each and every day is a good day to learn.

NDO · ‎12-12-2014

yes no problems:

ping 192.168.1.39
PING 192.168.1.39: 64 byte packets
64 bytes from 192.168.1.39: icmp_seq=0. time=0. ms
64 bytes from 192.168.1.39: icmp_seq=1. time=0. ms
64 bytes from 192.168.1.39: icmp_seq=2. time=0. ms

Eric Antunes · ‎12-12-2014

What about /var/adm/syslog/syslog.log?

Each and every day is a good day to learn.

eric_wilford · ‎12-14-2014

The "close_wait" socket is unrelated. You clearly have some mail issues. It would be best to test sendmail directly with a command such as:

# sendmail -v fretagi@mcel.co.mz < /etc/hosts

Sendmail will let you know exactly what it is trying to do on the screen... Since mailx should just be invoking sendmail in the background, this gets mailx out of the way. Also just test a name lookup of the target hostname with:

# nslookup mcel.co.mz

NDO · ‎12-14-2014

Hi

Please the output of the sendmail command is as follows:

sendmail -v  fretagi@mcel.co.mz < /etc/hosts
/dead.letter... Saved message in /dead.letter

and the nslookup gave me the following:

nslookup mcel.co.mz
Name Server:  intranete.headquarter.tmm
Address:  192.168.1.69

Trying DNS
Name:    mcel.co.mz
Addresses:  169.254.233.166, 192.168.7.5, 190.55.1.239, 192.168.0.9
          10.1.10.100

itc01[290]/ #

This is very strange, because all hp-ux servers (11) are now not sending emails, but all solaris 10 (2) are sending emails??

NDO · ‎12-14-2014

I have checked the contents of the dead.letter file:

more dead.letter
From root Mon Dec 15 09:15:37 2014
Return-Path: <root>
Received: (from root@localhost)
        by itc01.mcel.co.mz (@(#)Sendmail version 8.13.3 - Revision 1.003:: HP-UX11.31 - 8th December,2008/8.13.3) id sBF7FbB0014487
        for fretagi@mcel.co.mz; Mon, 15 Dec 2014 09:15:37 +0200 (SAST)
Date: Mon, 15 Dec 2014 09:15:37 +0200 (SAST)
From: root
Message-Id: <201412150715.sBF7FbB0014487@itc01.mcel.co.mz>

## Configured using SAM by root on Fri Sep 17 10:18:54 2010
# @(#)B.11.31_LRhosts $Revision: 1.9.214.1 $ $Date: 96/10/08 13:20:01 $
#
# The form for each entry is:
# <internet address>    <official hostname> <aliases>
#
# For example:
# 192.1.2.34    hpfcrm  loghost
#
# See the hosts(4) manual page for more information.
# Note: The entries cannot be preceded by a space.
#       The format described in this file is the correct format.
#       The original Berkeley manual page contains an error in
#       the format description.
#

10.100.48.10    itc01 itc01.mcel
10.100.48.11    prep01
10.100.48.12    prep02
10.100.48.13    prep03 prep03.mcel
127.0.0.1       localhost       loopback
10.1.20.23      mceldev mceldev.mcel

dead.letter: END

eric_wilford · ‎12-15-2014

So clearly we are not even trying to get the mail off the box. I suspect that you have one of:

1) DNS problem such as no MX record for "mcel.co.mz"

2) A permission or ownership issue on /var/mail or /var/mail/root

3) An incorrect sendmail.cf submit.cf or sendmail.cw file

I'd suggest a few simple tests such as:

# /sbin/init.d/sendmail stop

# /sbin/init.d/sendmail start

Check the screen for startup errors then check mail.log for sendmail errors

Check ownership and permissions on root's mailbox and verify there are no lock files:

# ll /var/mail |grep root

Try and send a simple mail message to root on the local system:

# sendmail -v root < /etc/hosts

Verify that the hostname is set correctly and correponds to one of the local IP addresses:

# hostname

# netstat -i

# netstat -in

You can also check sendmails debug outputs such as:

# sendmail -v -d 38.99 -d 8.99 fretagi@mcel.co.mz < /etc/hosts

Hmm... might also want to check to be sure you do not have a high load average which will cause sendmail to attempt to queue mail.

# uptime

Best of luck...

Categories

Company

Local Language

Forums

Discussions

Knowledge Base

Forums

Discussions

Knowledge Base

Forums

Discussions

Knowledge Base

Forums

Discussions

Forums

Discussions

Discussions

Forums

Forums

Discussions

Forums

Discussions

Forums

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Community

Resources

Other HPE Sites

Discussions

Forums

Blogs

CLOSE_WAIT HP-UX 11.31

CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31

Re: CLOSE_WAIT HP-UX 11.31