HP will not fix performance problem on 11.11 NFS-client

[Hartmut Lang]

We discovered a serious performance problem on our HPUX 11.11 NFS-clients.
Running our cross-assembler with debug option on a 11.11 box we observed runtimes to be about 100(!) times slower compared to runing the same program on a 10.20 box (both boxes with same HW). The cross-assembler creates an output file on a NFS-mounted directory.

HP support (asking the guys in the HPUX labs) could find out the reason for this poor performance. The cross-assembler issues a big number of write() and lseek(..., SEEK_END) system calls. On the 11.11 NFS-client each lseek()-call triggers the biod to flush its 32k write-cache to the server. This is not the case on 10.20 NFS-clients (i don't know about 11.0 NFS-clients).

The problem can be reproduced using the attached C-code. Creating and running an executable "nfstest" leads to the following results (running the executable on a nfs-mounted directory):

10.20 NFS-client:
> time nfstest 1
real 0.4
user 0.0
sys 0.2

11.11 NFS-client:
> time nfstest 1
real 35.2
user 0.0
sys 0.5

HP told me that they will not(!) fix this problem with a patch. The problem might(!) be fixed with later HPUX releases. Since the problem will not be fixed soon, he even could not give me a SR-number about this issue.
Why will this not be fixed?
HP does not expect that many customers are affected by this poor performance. And the fix is not easy.

So i ask the forum:
- is there anybody, who observed the same performance problems on 11.11 NFS-clients?
- what do you think about HP decision not to fix this issue?

Hartmut

[Alex Glennie]

Hartmut,

Well it would appear the problem is with lseek and was reported to HP end of last year, I've found an SR relating to this and quoting your exact eg code ? : fyi : SR = CR# JAGad958000.

It would be interesting to find out from other ITRC users whether this occurs on non hpux NFS clients eg Solaris.

I suspect the problem is not that HP will not fix the problem in a patch but that it's not possible to at least without adversely affecting other functionality or at least without significant changes which will take time and need to be tsted etc ...

My advise would have to be to get back to HP quoting the above SR and seeking an update to confirm this is still the case. You may not like the answer but at least you'll have a better understanding of why it was given.

[Hartmut Lang]

Alex,

thanks for your response.
It was HP support who told me that there is no CR (or SR) number available. I asked for such a number. He could not give me one.
But he told me the same thing: since it is a significant change (needs lot of testing, ...) and only few customers are affected, the decision was not to implement it in a patch.

I did tests with the same program on Linux 2.4.10, SunOS 5.8, HPUX10.20, HPUX11.11.
Only HPUX11.11 showed the bad performance problem.

Hartmut

[Alex Glennie]

I can only say publically so much via these forums ... there's definitely a SR raised and the issue appears to be under investigation still. I suspect the problem will occur on 11.00 as I *suspect* it's due to the version of NFS as opposed to the O/S ie ver 3.

I'd still suggest getting an update from HP support on the status of the above SR if you are still unhappy.

[Ravi_8]

Hi,

I have few HP machines (L-class) came from HP with pre-installed 11i, while powering on (for the first time at our site) all machines hangs while checking for NFS clients.

I have to reboot (power off), get into single user mode, turn off NFS-SERVER/CLIENT flags(/etc/rc.config.d/nfsconf) then boot again.
There is a bug wrt NFS in 11i.

[Brian Hackley]

Hartmut,

I looked up that status of CR # JAGad95800, which is listed as SR # 8606226738 (SR is the customer viewable doc #, CR is that HP Lab bug id) and confirmed the information in the discussion thread. I know it sounds like something that should be simple to resolve, belive me! Unfortunately it is not, so what I suggest to you is that escalation of the issue should be done through the HP Response Center and/or your local HP Customer Support contacts, rather than through the ITRC Forums.
Have you considered running with 0 biod's and observe the results?? Edit the /etc/rc.config.d/nfsconf file NUM_NFSIODS=0 and then reboot. I wish I had better news, suggestions or coding changes beyond this!!

Ravi as far as your specific issue is concerned, you might want to break that one out into a seperate discussion thread...

Hope this helps,
-> Brian Hackley

[harry d brown jr]

Have you tried using cifs9000 (samba) to replace NFS?

live free or die
harry

[Hartmut Lang]

Many thanks for your responses!

Alex,
i still have to call support about the status of this SR. But since support told me last week, that this issue will not be fixed soon (or at all), i did not expect that i should call again this week to check the current status.

Ravi,
i think your problem is not related to this issue.

Brian,
thank you giving me the SR#.
Is there a way for me to lookup the related document? I could not find it on ITRC.

I did try runing 0 biods, with same (bad) performance.

The reason why i posted in the forum was to find out, if maybe other HPUX11.11 users are affected. I think they are! But they might not be so heavily affected as we are (because their application makes less frequent lseek() calls).

Harry,
i will look into using CIFS instead of NFS. But since out development environment is build around NFS, i propably can not change. I think it will be easier to stay with HPUX10.20 or switch to Linux or Solaris.


Hartmut

[sven verhaegen]

Hi , I have to agree with Brian on his explanation about the fix , indeed there might be some very good reasons why the labs cannot implement the fix at first however I do feel that if your environment is enough at peril you might just return to your local responce center and ask for an escalation ... if it does affect your bussiness seriously and if you can prove that you stand to loose money on this HP might be entraved to see implementation differently and still consider pushing the JAG forward .. it doesn't hurt to try

[Hartmut Lang]

Sven,

our point is not that "we are loosing money" if this issue is not fixed.

The point is, that i have to tell my manager "well, using the HPUX 11.11 machine you have to wait 4min everytime you run the assembler, on the Linux box it will complete in less than 2 seconds".

The point is that we have to decide if HPUX is the "right" development environment or not. And we have some troubles with tools not being available for HPUX, expensive SW and HW, unfixed performance issues(!),...

There are many pieces, which lead to a decision to switch to Solaris or Linux, or stay with HPUX.


Hartmut