Very Stange Problem - HP-UX 11.11

[Lee Harris_5]

Hi,

I hope someone can help me because this is a very strange problem. I have noticed that it seems that I am not able to run commands from within certain filesystems without them hanging.

It seems that the problems only occur when my $PWD is a filesystem that resides on external disk. I cannot run commands such as...

rm (with the -rf options)
which
man

...these just hang, BUT, these commands...

ls
cd
rm (with no options)
top

...all run fine.

However, the problems do not appear to be present if $PWD is a filesystem on internal disk.

There are no disk related errors in syslog, or dmesg, ioscan looks fine, the load on the box is very low, iowait is very low (mostly 0 with the occasional 2-5% WIO).

I'm totally confused by this behaviour. Does anyone have any clues for me?

Thanks - Lee

[OFC_EDM]

Last 2 times I saw this had similar load symptoms....but the disk utilization was high.

There was no consistency in which commands were slow to respond.

Came down to someone accidentally kicking off a full backup during production hours. Killed the backup, not the admin ;) , and all returned to normal.

A 2nd time it was a process spawning hundreds of smaller processes. Load was low but number of processes running were insane. Killed jobs and system returned to normal.

[James R. Ferguson · Accepted solution]

Hi Lee:

Any NFS mounts involved?

Is '.' (the dot) in your PATH and where? If so, the beginning? the end?

When did yuo first observe this? What changed just before you noticed this behavior?

Regards!

...JRF...

[Lee Harris_5]

Thanks for the replies guys... as far as I know nothing significant changed before the behaviour began.

The filesystems I'm working from within, e.g. the ones I'm in when I issue the commands that hang are local (on external SCSI disks) and not NFS mountpoints. However we are running an NFS server on this machine that is exporting some of the affected filesystems, but not all.

dot (.) is not in my $PATH anywhere.

[Steven E. Protter]

Shalom,

JRF correctly points out that this behavior is typical of stale NFS mounts. Might there be a softlink to an NFS mount buried in there.

The alternative is that there is a bad second of disk the file system is sitting on or an open file handle with an i/o lock.

fuser -cu /filesystem

SEP

[Lee Harris_5]

Thanks for your help... I never did get to the exact cause of the problem, and we had to reboot the server anyway, at which point once it came back up it was behaving itself again. I'm guessing the problem was to do with the NFS services running on the box.