lsof 4.81 issue

[Court Campbell]

I have lsof installed on an 11.23 PA machine. When I run the command it just sits there and never returns.In glance it says the process id waiting on PIPE. Anyone else seen this issue?

[Avinash20]

Are you using any specific flags with lsof
I have faces a issue with lsof hang, when I used -O flag

[Court Campbell]

I am running lsof +aL1 /var. Sometimes we have oracle processes that use up all the inodes making the filesystem seem full. But when I run the command it just sits. I have been waiting close to 45 minutes for something to return. For grins I even compiled from source to no avail.

[Avinash20]

Check if lsof works fine with other filesystem

[Patrick Wallek]

I just installed lsof 4.81 on an 11.23 PA server (an old A500) and it seems to work fine.

I tried running 'lsof +aL1 /var' and it returned almost instantly.

The lsof that I installed was downloaded from the Internet Express bundle for 11iv2 here:

http://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=HPUXIEXP1123

[Court Campbell]

I have three 11.23 PA systems. Two of them have this issue. And one is working fine. I know that it should not be taking this long, hence my query.

[Court Campbell]

Running tusc shows it going to sleep when it gets to pstat(PSTAT_SOCKET).

[Armin Kunaschik]

Maybe you have some name resolution issue (DNS/NIS)?

In additon if you have lots of memory and lots of processes and open connections this might delay the output substantialy. On a busy 24CPU/256GB SAP system I have to wait severy minutes for lsof to complete (I'm using 4.78 from HP-UX internet express).

To be curious: Did it work with an earlier lsof version? Do you really need the brand new 4.81?

My 2 cents,
Armin

[Court Campbell]

All right. I have reopened this thread. HP support has been useless. The problem lies with PSTAT and/or libc.2. When I run lsof with gdb and do a backtrace, I get this:

(gdb) where
#0 0x7b00ab68 in pstat+0x10 () from /usr/lib/libc.2
#1 0x7afe0174 in __pstat_getsocket64+0x34 () from /usr/lib/libc.2
#2 0x6180 in read_sock+0x30 ()
#3 0x5fe0 in gather_proc_info+0x1098 ()
#4 0xe3f4 in main+0x2014 ()

I have installed every known pstat patch out there. I have installed quite a few networking related patches also. Any ideas? Dennis Handley, are you out there?

P.S. Please do not post any lame answers aksing about DNS, etc. I am looking for real answers.

[Steven E. Protter]

Shalom,

That is a pretty typical issue. I've seen it on several OS releases.

My way around it was to compile the code myself.

That didn't always work, but sometimes it did.

SEP

[Court Campbell]

Thanks SEP, but I have already done that. I have also talked with Vic Abell and he is pointing towards HP implementation of PSTAT as the issue.

[Dennis Handly]

I get the same stacktrace on IPF but I don't get any hangs with: lsof +aL1 /var

It doesn't seem like pstat_getsocket should be waiting for PIPE.

>HP support has been useless.

In what way? Have you given them a small test case that calls pstat_getsocket and hangs?

Do you know which socket it is hanging on?

Try finding which PID has the socket? Loop through all of the PIDs with "lsof +aL1 /var -p PID1":
for P in $(UNIX95=EXTENDED_PS ps -e -opid=); do
lsof -p $P +aL1 /var
done

[Court Campbell]

> In what way?

Every time I call support I get dumped to a call center in India or Costa Rica. Then I have to ask to be sent to tier 2 support. Then the guys at that level aren't software engineers. Basically, I am not able to talk to the correct people.

[Court Campbell]

It hangs when it hit the pid for /etc/vx/bin/vxconfigbackupd

[Court Campbell]

Oops, wrong pid. It's actually:

/opt/java1.5/jre/bin/PA_RISC2.0/java -DXPPA=1 -Djava.net.preferIPv4Stack=true -cp classes/xpdevicemap.jar:classes/bccaservice.j

Looks like the xp performance advisor agent.

[Court Campbell]

I stopped the process and now lsof is working fine. I wonder who I need to contact at HP to get them to look at this.

[erics_1]

Court,

Ask that your case be sent to the backline languages group. Chances are you're being dumped into backline sys admin.

Hope you get somewhere with it.

Regards,
Eric

[Court Campbell]

Dennis. I need you to reply. You deserve a bunny for this.

[Dennis Handly · Accepted solution]

>I am not able to talk to the correct people.

I would have thought HP-UX support would be better than this.

>Oops, wrong pid. It's actually:

I probably should have had this in the loop:
UNIX95=EXTENDED_PS ps -x -p $P -opid= -oargs=

>need you to reply. You deserve a bunny for this.

Here you go.

>erics: Ask that your case be sent to the backline languages group.

This has nothing to do with languages, this is more kernel or libc. When they ask me, I would just have to them to move it along. :-)

[Dennis Handly]

>Looks like the xp performance advisor agent.

Now that you know the application, it should be a simple matter to write a program to look through the files and show pstat_getsocket hangs. ;-)

[Court Campbell]

lsof was stuck when making calls against the pid for xppa. I was able to find the issue with the help Vic Abell (lsof Author) and Dennis Handley. Thanks Dennis for pointing me in the right direction. I never thought about it getting stuck on a certain pid.

P.S. I started xppa up again and it worked. The process must have been having issues.

[Dennis Handly]

>I never thought about it getting stuck on a certain PID.

It doesn't really get stuck on a PID, it gets stuck on a file/socket. The stack trace gives a clue on how to triage, first isolate it to a PID.
And you were lucky it was a specific application.