HPUX ftp Problem - 421 Service not available, remote server has closed connection

[curtisk_1]

Hi,

I am currently experiencing a problem with one of my HPUX servers when using ftp.

We have a daily application script that simply ftp's to a external ftp site and then put's approx 2000 pdf files.

This script has recently started to fail with the below message.

421 Service not available, remote server has closed connection

I have tried this process manually and I still get the same error.

I have also tried the same procedure to a different external ftp server and this also errors.

If I try the same procedure from one of the other unix nodes within my HPUX cluster it works fine.

Very frustrating..

I have spoken to both firewall teams from our company and the external and we believe the problem to be with the HPUX machine and not the firewalls or external ftp servers.

Do you have any helpfull advice on what could cause this kind of problem within HPUX?

I have checked the syslog and the ftp log but can find no errors being reported.

when doing a netstat -s i do the the below

10540 connect requests dropped due to full queue
2393372 connect requests dropped due to no listener
0 suspect connect requests dropped due to aging
0 suspect connect requests dropped due to rate

However on test we did try reducing the tcp_conn_request_max value right down and creating lots of connection but could not replicate the issue.

On live this is set to 4096 (default) tcp_conn_request_max

If I check for established connections I dont think we are anywhere near a limit

ndd -get /dev/tcp tcp_status | grep EST | wc -l

Averages around 1500

The firewall logs are showing that our HPUX machine is issuing a TCP ZeroWindow but for a process that has worked for months I am struggling to figure out what the issue is.

Any assistance you could give me would be greatly appreciated.

[tkc]

have u refer to this document already?

http://docs.hp.com/en/5991-6480/ch05s02.html

[curtisk_1]

Thanks tkc but yes I have tried all the usual steps.

Over the weekend we are goign to try failing over the firewalls as I am convinced this is not a problem with my HPUX server.

I will keep you posted

[whiteknight]

Hi

Verify that /var/adm/inetd.sec does not contain an ftp entry of either
deny or allow. When you allow one user, you deny all other users.
For troubleshooting purposes you could rename /var/adm/inetd.sec to
/var/adm/inetd.sec.old. inetd.sec is not needed unless you have
a need for tightened security beyond login verification.

WK

[tkc]

i would agree with what white knight has said. do consider his option.

[skt_skt]

i would suspect the defualt time out setting for the ftp client and server. it is 2 hours and 15 minutes respectively if my memorybis true..

man ftpd

[Steven Schweda]

So, if I'm interpreting this correctly, your
HP-UX system is the FTP client

> This script has recently started to fail
> with the below message.
>
> 421 Service not available, remote server
> has closed connection

Can you provide a short failing script?

What is the failing operation, initial
connection, log-in, PUT, ...?

What are the remote systems? (OS, FTP
server, ...?) Any useful log entries at the
other end?

> If I try the same procedure from one of the
> other unix nodes within my HPUX cluster it
> works fine.

Do you mean using a different HP-UX system as
the FTP client (that is, running the script
on a different HP-UX system), or using the
first HP-UX system to send files to another
of your HP-UX systems (instead of to a remote
system)?

Passive mode can help with some firewall
problems.

[Bill Hassell]

I think you can nail down the problem very quickly by doing a Wireshark trace. Since it fails immediately, the trace will definitively show where the connection was terminated. You may have to move from the client LAN into the middle of the routers to show the failure.

[Steven Schweda]

> Since it fails immediately, [...]

Who said that it failed "immediately"?

Did I miss something (again)? All I saw was:

> This script has recently started to fail
> [...]

About 421, the RFC says, "This may be a reply
to any command [...]", which is why I asked
when it happened.

Without knowing anything about the FTP server
involved, it may be tough to divine the
reason it would return a 421 status.

Unless I misinterpreted this whole thing, and
the FTP _server_ is actually on an HP-UX
system, some of the suggestions offered here
may not have great value.

As is often the case, a clearer explanation
of the problem would be helpful. Minor
details (like who's who, who does what, and
what happens then) can often help to reduce
the bafflement.

[curtisk_1]

Further Tests run this weekend suggest that the problem is with our firewalls and not the HPUX environment.

Thanks for all your advice. I will confirm the full resolution once we have fully identified it.

[Steven Schweda]

> I will confirm the full resolution once we
> have fully identified it.

I'd settle for a clear description of the
_problem_.