topic Re: Bizarre lockd behaviour in Operating System - HP-UX

Bizarre lockd behaviour

Pyers Symon — Thu, 08 Oct 2009 11:04:38 GMT

We have an issue where we are getting very strange behavior between an HP-UX 11.11 NFS client and an Soalris 8 server. In essence it seems that the Solaris lockd is only responding intermittently to an rpcinfo call:

For example:

# rpcinfo -T tcp fid001 nlockmgr 1
rpcinfo: RPC: Timed out
program 100021 version 1 is not available

To complicate issues this is part of a Veritas cluster .......

However a few second later:

# rpcinfo -T tcp fid001 status 1
program 100024 version 1 ready and waiting

Further investigation shows that the response is very slow (when it works) but is always below the 10 sec timeout of rpcinfo. truss & netstat on the Solaris box indicates that a connection is being made in all cases but for some reason the server side is not responding correctly at the lower level...

Any thoughts?

I suspect that this is at teh Sun end ...

Re: Bizarre lockd behaviour

Dave Olker — Fri, 09 Oct 2009 02:24:19 GMT

Your two examples were testing two different RPC programs. The first was lockd and the second was statd. Does the same RPC daemon respond differently at different times?

As for which system is at fault, the best way to determine that would be to collect a network trace on *BOTH* systems. A trace on the HP-UX system should show when the packet is sent and how long it takes to receive a reply. A trace on the Sun system should show when the request arrives and how long it takes for it to respond. In many cases the client and server traces can be misleading on their own so it helps to get a trace from both systems and compare them.

Regards,

Dave

Re: Bizarre lockd behaviour

Pyers Symon — Fri, 09 Oct 2009 05:53:27 GMT

Sorry Dave (hi again!)

That was my fault .. I didn't check what I wrote. Both examples should have been from lockd ... (I was checking statd at the same time)

This is what is should have looked like ...

fid001# rpcinfo -T tcp fid001 nlockmgr 1
program 100021 version 1 ready and waiting
fid001# rpcinfo -T tcp fid001 nlockmgr 1
rpcinfo: RPC: Timed out
program 100021 version 1 is not available

I will arrange a network trace for you ...

Re: Bizarre lockd behaviour

Pyers Symon — Mon, 12 Oct 2009 07:11:22 GMT

Turned out it was the number of lockd threads started on the server that was the issue. The system was set at the default of 20 (dec) which was being exceeded.