HP-UX error 232 ECONNRESET Connection reset by peer

[Matthias Danzer]

Hi folks,
we are running several oracle databases (9i/10g) on two node rp7410 HP-UX 11.11 MC/SG Cluster v. 11.16.

We have 2 databases running on one node called SPCP and TDSP. There is a listener named LSNRSPCP for the SPCP database listening on ip 1xx.xx.xx.11 and another listener named LSNRTDSP for the TDSP database
listening on ip 1xx.xx.xx.17.

From time to time we get the error that the SPCP database would not connect to TDSP database using the right connect string of the tnsnames. The SPCP tries to connect to a third database on the same machine. But the connect string is right.
So the source database gets the following error:

ORA-12505: TNS:LISTENER DOES NOT CURRENTLY KNOW OF SID GIVEN IN CONN

Which is right because the conncetion is made to the wrong database. This although the connection data of the tnsnames is correct. The problems couldn't be reproduced. Sometimes it works, sometimes not...


A of the listener logs was send to oracle. They found the following hp-ux error:

HP-UX error 232 ECONNRESET Connection reset by peer

...but it is not useful.

The DNS resolution works fine, tnsnames is right....

What is wrong?
How could I setup a useful trace?

Regards
Matthias

[Steven E. Protter]

Shalom,

You may have an inconsistency in your local DNS server.

I'd up the databases for enahnced logging and confirm the connect is going to the wrong system. Then track down the DNS issue.

Start with changes to /etc/resolv.conf

You may be able to identify the erant server.

SEP

[Matthias Danzer]

Hello,

I checked the /etc/resolv.conf

domain germany.xxxx.com
search germany.xxx.com

nameserver 1xx.xx.xx.xx # Primary DNS server
nameserver 172.xx.xx.xx # Secondary DNS server


nsswitch.conf
hosts: files [NOTFOUND=continue] dns
netgroup: files [NOTFOUND=return UNAVAIL=continue]
networks: files [NOTFOUND=return UNAVAIL=continue]
protocols: files [NOTFOUND=return UNAVAIL=continue]
publickey: files [NOTFOUND=return UNAVAIL=continue]
rpc: files [NOTFOUND=return UNAVAIL=continue]
services: files [NOTFOUND=return UNAVAIL=continue]

Entries in /etc/hosts are fine.

I have tested these through nslookup with short name, full qualified name and ip adress.

All works fine.

[Eric Antunes]

Hi Matthias,

Was this working or is this a recent issue?

Did someone change the startup scripts?

Can you post the $ORACLE_HOME/network/admin/listener.log or $ORACLE_HOME/network/admin/.log?

Finally, check Metalink Note 374624.1

Best Regards,

Eric Antunes

[Matthias Danzer]

Hi the configuration worked for four years now without any problems. After a HP-UX Patche Session incl. MC/SG. The Metalink doc does not match the problem.

[Oviwan]

Hey

Can you post the new installed patch #?

[Eric Antunes]

Hi Matthias,

This must be an issue caused by one or more of those HP-UX network patches (PHNE_...).

Best Regards,

Eric Antunes

[Matthias Danzer]

...yeap. The thing are the patches. But which patch????

I have uploaded the patch list.

[rick jones]

Connection reset by peer could mean a failure in application layer connection tear-down handshake. If the remote side calls shutdown(SHUT_RDWR) or close() and the local side happened to send data it will trigger an ReSeT (RST) segment from the remote.

It could also be caused by some other system usurping the remote IP address - local system sends traffic, arrives at the usurper, who finds no corresponding TCP connection state and an RST is send.

DNS flapping would, I expect show-up as connection _refused_ not connection reset because IP addresses are looked-up by apps at time of connect().

[Eric Antunes]

Hi Matthias,

Must be a PHNE patch. Did you followed all the special instructions? Patch PHNE_35508, for example, as those special instructions...

Best Regards,

Eric Antunes

[Eric Antunes]

Patch PHNE_35865 has a very important special instr.:

"Special Installation Instructions:
PHNE_27796:
After this patch is installed, *ONLY* if you want to
recompile and/or relink the applications that use libnsl.1
APIs, link the application explicitly with libnsl.1. This
is applicable only in the case of applications expecting
to resolve libnsl.1 symbols through libnss_dns.1.
Not linking these applications with libnsl.1 will cause the
linker to report unresolved symbols, as libnss_dns.1 no
longer resolves libnsl.1 entry points."

Best Regards,

Eric