trouble with remsh and rcp hanging

[Redneck]

I'm having trouble with remsh and rcp jobs hanging. 

foobar% uname -a
HP-UX blackpea B.11.31 U ia64 0372040118 unlimited-user license
foobar% swlist -l bundle | grep '\-OE'
HPUX11i-VSE-OE B.11.31.1109 HP-UX Virtual Server Operating Environment

There are some hanging processes:

id1    17327  1303  0 01:36:54 ?        00:00:00     tcsh -c rcp -r -d -t /some/directory

      -3 17334 17327  1 01:36:55 ?        00:00:00       <defunct>

id2  19841  1303  0  Feb 17  ?        00:00:00     tcsh -c rcp -r -t /another/directory

      -3 19851 19841  0  Feb 17  ?        00:00:00       <defunct>

id2  29916  1303  0  Feb 17  ?        00:00:00     tcsh -c rcp -r -t /some/other/directory

      -3 29925 29916  0  Feb 17  ?        00:00:00       <defunct>

id2  10419  1303  0  Feb 16  ?        00:00:00     tcsh -c rcp -r -t /some/third/directory

      -3 10426 10419  0  Feb 16  ?        00:00:00       <defunct>

id2  16290  1303  0  Feb 17  ?        00:00:00     tcsh -c rcp -r -t /some/fourth/directory

      -3 16299 16290  0  Feb 17  ?        00:00:00       <defunct>

(Basically id1 and id2 remsh and rcp to the localhost.)

Here are some other processes that are hanging:

   root 17235  1303  0  Feb 15  ?         0:00 remshd

    root 18449  1303  0  Feb 16  ?         0:00 remshd

    root  5330  1303  0  Feb 15  ?         0:00 remshd

    root   105  1303  0  Feb 16  ?         0:00 remshd

    root 14362  1303  0  Feb 16  ?         0:00 remshd

    root 14973  1303  0  Feb 16  ?         0:00 remshd

    root 17382  1303  0  Feb 17  ?         0:00 remshd

tusc indicates that the remshd's are all sleeping.

The latest r-commands patch is installed: PHNE_38419

foobar% what /usr/lbin/remshd

/usr/lbin/remshd:

         Copyright (c) 1983, 1988 The Regents of the University of California.

         $Revision: @(#) src R11.31_BL2008_0918_1 PATCH_11.31 PHNE_38419 

I upgraded from 11.23 to 11.31 this past week. This was a scratch install onto a new disk, with the development environment copied onto the new disk. 

Any thoughts on what might be amiss with this system?

thanks,

 red

[Redneck]

One other useful point: not all remsh and rcp commands are having, just some of them.

[Redneck]

^having^hanging

[Dennis Handly]

>tusc indicates that the remshds are all sleeping.

What does it show for the tcsh processes?

What happens if you use a real shell instead?  (ksh or sh)

I've had cases of the scummy C shell hanging until I redirected stdin to /dev/null.