rpcbind: cannot accept connection

[Sean Dale]

The following error repeats many times in the syslog on our production server:

rpcbind: cannot accept connection: bad call sequence number (current state 4)

I cannot figure out why. We used to have an nfsmount to a remote server, but, it is no longer mounted. It's not exported from the remote server any longer, either.

Is there any way to figure out what is making the rpc call so I can remove the errors from the syslog? Thanks

[RAC_1]

rpcinfo -p "host_name"

Does any of the progrma look suspicious??
. Also once you get port number, you can run lsof/netstat to know what program is listening on that port.

[Sean Dale]

this is the output : The only thing that looks weird to me is the one listed at port 822

program vers proto port service
100000 4 tcp 111 rpcbind
100000 3 tcp 111 rpcbind
100000 2 tcp 111 rpcbind
100000 4 udp 111 rpcbind
100000 3 udp 111 rpcbind
100000 2 udp 111 rpcbind
100024 1 tcp 49152 status
100024 1 udp 49154 status
100021 1 tcp 49153 nlockmgr
100021 1 udp 49155 nlockmgr
100021 3 tcp 49154 nlockmgr
100021 3 udp 49156 nlockmgr
100021 4 tcp 49155 nlockmgr
100021 4 udp 49157 nlockmgr
100020 1 udp 4045 llockmgr
100020 1 tcp 4045 llockmgr
100021 2 tcp 49156 nlockmgr
100068 2 udp 49160 cmsd
100068 3 udp 49160 cmsd
100068 4 udp 49160 cmsd
100068 5 udp 49160 cmsd
100083 1 tcp 49157 ttdbserver
805306352 1 tcp 822
100005 1 udp 49174 mountd
100005 3 udp 49174 mountd
100005 1 tcp 49195 mountd
100005 3 tcp 49195 mountd
100003 2 udp 2049 nfs
100003 3 udp 2049 nfs
100003 2 tcp 2049 nfs
100003 3 tcp 2049 nfs

[RAC_1]

Identify the program associated with that port.

nestat -an | grep 822
with lsof
lsof -i tcp:822

Anil

[Sean Dale]

# netstat -an | grep 822
tcp 0 0 172.x.x.x.822 172.x.x.x.49175 ESTABLISHED

tcp 0 0 172.27.16.135.49175 172.x.x.x.822 ESTABLISHED

tcp 0 0 *.822 *.* LISTEN


I'm still not sure what's happening here. It looks like the server is listening on that port and connecting to itself? I am not able to determine whan program is associated with this port (assuming this is the port causing the trouble).

[Mel Burslan]

lsof -i tcp:822

should give you more information about this process as anil suggested.

[Sean Dale]

#lsof -i tcp:822
sh: lsof: not found.

Looks like I do not have this utility. Is it something I can install without interuppting the server (it's production)?

[Srini Jay]

Hi Sean Dale,
You can download download the binary depot or source code of lsof from http://hpux.cs.utah.edu/hppd/hpux/Sysadmin/lsof-4.74/

Well, it shouldn't interrupt anything (atleast it didn't when I installed lsof-4.64 some time back!!)

[Mel Burslan]

just make sure your OS kernel width and get the right bit size version for you.

to determine which kernel width your OS is, run command

getconf KERNEL_BITS

if it is 32, look for 32 bit version of lsof. It is not available from porting archive anymore but you still can find it via some googling.

[Sean Dale]

I will download the 64-bit version. Thank you for the link and assistance. I have a development box I can install onto first to make sure it doesn't cause a reboot.

[RAC_1]

There should be an entry in /etc/services and /etc/inetd.conf. (Though not necesserry)

Anil

[Sean Dale]

I did a search for 822 in both files and come up with nothing. Either that is not the port that is causing the rpcbind errors, or, it is, but, I cannot locate it anywhere. This is starting to drive me crazy...

[Mel Burslan]

Port 822 does not seem to be a registered port number according to this list I reference from time to time:

http://www.colasoft.com/resources/ports_list.php

If it does not get listed in /etc/services, it does not mean that it doesn't exist in your system. It is just another ill-coded program listening on this port, rogue application for the lack of a better term.

Good behaving apps, list themselves in /etc/services and inetd.conf and get launched by inetd to minimize the load they put on the processor. But it looks like you application is in a daemon mode and continuously litening to port 822. Hope that it is not something malicious, planted by a disgruntled ex-employee or something. This kind of stuff always make me feel uneasy

[Sean Dale]

# lsof -i tcp:822
lsof: WARNING: no block devices found
lsof: WARNING: /home/root/.lsof was updated.

It looks like I may have to open a sw case w/ HP.