topic lsof 4.81 issue in Operating System - HP-UX

lsof 4.81 issue

Court Campbell — Tue, 03 Mar 2009 14:16:35 GMT

I have lsof installed on an 11.23 PA machine. When I run the command it just sits there and never returns.In glance it says the process id waiting on PIPE. Anyone else seen this issue?

Re: lsof 4.81 issue

Avinash20 — Tue, 03 Mar 2009 14:48:49 GMT

Are you using any specific flags with lsof
I have faces a issue with lsof hang, when I used -O flag

Re: lsof 4.81 issue

Court Campbell — Tue, 03 Mar 2009 14:54:15 GMT

I am running lsof +aL1 /var. Sometimes we have oracle processes that use up all the inodes making the filesystem seem full. But when I run the command it just sits. I have been waiting close to 45 minutes for something to return. For grins I even compiled from source to no avail.

Re: lsof 4.81 issue

Avinash20 — Tue, 03 Mar 2009 15:29:13 GMT

Check if lsof works fine with other filesystem

Re: lsof 4.81 issue

Patrick Wallek — Tue, 03 Mar 2009 15:34:17 GMT

I just installed lsof 4.81 on an 11.23 PA server (an old A500) and it seems to work fine.

I tried running 'lsof +aL1 /var' and it returned almost instantly.

The lsof that I installed was downloaded from the Internet Express bundle for 11iv2 here:

http://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=HPUXIEXP1123

Re: lsof 4.81 issue

Court Campbell — Tue, 03 Mar 2009 15:42:02 GMT

I have three 11.23 PA systems. Two of them have this issue. And one is working fine. I know that it should not be taking this long, hence my query.

Re: lsof 4.81 issue

Court Campbell — Tue, 03 Mar 2009 16:24:33 GMT

Running tusc shows it going to sleep when it gets to pstat(PSTAT_SOCKET).

Re: lsof 4.81 issue

Armin Kunaschik — Wed, 04 Mar 2009 10:08:08 GMT

Maybe you have some name resolution issue (DNS/NIS)?

In additon if you have lots of memory and lots of processes and open connections this might delay the output substantialy. On a busy 24CPU/256GB SAP system I have to wait severy minutes for lsof to complete (I'm using 4.78 from HP-UX internet express).

To be curious: Did it work with an earlier lsof version? Do you really need the brand new 4.81?

My 2 cents,
Armin

Re: lsof 4.81 issue

Court Campbell — Thu, 05 Mar 2009 19:50:35 GMT

All right. I have reopened this thread. HP support has been useless. The problem lies with PSTAT and/or libc.2. When I run lsof with gdb and do a backtrace, I get this:

(gdb) where
#0 0x7b00ab68 in pstat+0x10 () from /usr/lib/libc.2
#1 0x7afe0174 in __pstat_getsocket64+0x34 () from /usr/lib/libc.2
#2 0x6180 in read_sock+0x30 ()
#3 0x5fe0 in gather_proc_info+0x1098 ()
#4 0xe3f4 in main+0x2014 ()

I have installed every known pstat patch out there. I have installed quite a few networking related patches also. Any ideas? Dennis Handley, are you out there?

P.S. Please do not post any lame answers aksing about DNS, etc. I am looking for real answers.

Re: lsof 4.81 issue

Steven E. Protter — Thu, 05 Mar 2009 20:11:51 GMT

Shalom,

That is a pretty typical issue. I've seen it on several OS releases.

My way around it was to compile the code myself.

That didn't always work, but sometimes it did.

SEP

Re: lsof 4.81 issue

Court Campbell — Thu, 05 Mar 2009 20:16:00 GMT

Thanks SEP, but I have already done that. I have also talked with Vic Abell and he is pointing towards HP implementation of PSTAT as the issue.

Re: lsof 4.81 issue

Dennis Handly — Fri, 06 Mar 2009 03:35:05 GMT

I get the same stacktrace on IPF but I don't get any hangs with: lsof +aL1 /var

It doesn't seem like pstat_getsocket should be waiting for PIPE.

>HP support has been useless.

In what way? Have you given them a small test case that calls pstat_getsocket and hangs?

Do you know which socket it is hanging on?

Try finding which PID has the socket? Loop through all of the PIDs with "lsof +aL1 /var -p PID1":
for P in $(UNIX95=EXTENDED_PS ps -e -opid=); do
lsof -p $P +aL1 /var
done

Re: lsof 4.81 issue

Court Campbell — Fri, 06 Mar 2009 13:23:08 GMT

> In what way?

Every time I call support I get dumped to a call center in India or Costa Rica. Then I have to ask to be sent to tier 2 support. Then the guys at that level aren't software engineers. Basically, I am not able to talk to the correct people.

Re: lsof 4.81 issue

Court Campbell — Fri, 06 Mar 2009 13:38:53 GMT

It hangs when it hit the pid for /etc/vx/bin/vxconfigbackupd

Re: lsof 4.81 issue

Court Campbell — Fri, 06 Mar 2009 13:47:48 GMT

Oops, wrong pid. It's actually:

/opt/java1.5/jre/bin/PA_RISC2.0/java -DXPPA=1 -Djava.net.preferIPv4Stack=true -cp classes/xpdevicemap.jar:classes/bccaservice.j

Looks like the xp performance advisor agent.

Re: lsof 4.81 issue

Court Campbell — Fri, 06 Mar 2009 13:59:11 GMT

I stopped the process and now lsof is working fine. I wonder who I need to contact at HP to get them to look at this.

Re: lsof 4.81 issue

erics_1 — Fri, 06 Mar 2009 14:32:40 GMT

Court,

Ask that your case be sent to the backline languages group. Chances are you're being dumped into backline sys admin.

Hope you get somewhere with it.

Regards,
Eric

Re: lsof 4.81 issue

Court Campbell — Fri, 06 Mar 2009 14:43:04 GMT

Dennis. I need you to reply. You deserve a bunny for this.

Re: lsof 4.81 issue

Dennis Handly — Fri, 06 Mar 2009 20:49:55 GMT

>I am not able to talk to the correct people.

I would have thought HP-UX support would be better than this.

>Oops, wrong pid. It's actually:

I probably should have had this in the loop:
UNIX95=EXTENDED_PS ps -x -p $P -opid= -oargs=

>need you to reply. You deserve a bunny for this.

Here you go.

>erics: Ask that your case be sent to the backline languages group.

This has nothing to do with languages, this is more kernel or libc. When they ask me, I would just have to them to move it along. :-)

Re: lsof 4.81 issue

Dennis Handly — Fri, 06 Mar 2009 20:56:55 GMT

>Looks like the xp performance advisor agent.

Now that you know the application, it should be a simple matter to write a program to look through the files and show pstat_getsocket hangs. ;-)