topic Re: NFS transfer times in Operating System - OpenVMS

NFS transfer times

Brian Reiter — Thu, 27 Oct 2005 13:53:44 GMT

Hi Folks,

Heres the situation. 2 DS10s (1 Server 1 Statation) running VMS 7-3-2 TCPIP 5-4 ECO 5. The DS10 AlphaStation is configured as the NFS server, the other DS10 is the client. During a test the client was able to transfer data from the server at a rate of 5 Mbytes per second, however transferring data to the server went at 0.9 Mbytes per second. At the time of the test there was no other LAN or disk activity.

The network i/f is at 100Mbs and both sides have autonegotiated to a 100 full. I suppose the questions are:

1) Is this normal?
2) If this behaviour is not normal what on the server could cause this discrepancie?

I'm tied to using NFS in this situation, the customer is adamant that we cripple our systems with unix stuff.

Regards

Brian

Re: NFS transfer times

Arch_Muthiah — Thu, 27 Oct 2005 14:45:34 GMT

Hi Brian,

Please make sure the send and receive queue sizes and network state are ok using

TCPIP's> "netstat" command.

poor network configuration both in client and server side may be the reason poor nfs performance.

Re: NFS transfer times

Ian Miller. — Thu, 27 Oct 2005 15:32:27 GMT

monitor the system running the NFS server.
MONITOR IO and MONITOR FCP

Is the file being extended in small amounts each time? I think will show in FCP activity?

Is highwater marking on - will show as erase activity?

Is the target disk heavily fragmented -lots of window turns?

Re: NFS transfer times

Volker Halle — Fri, 28 Oct 2005 01:12:20 GMT

Brian,

consider to test the same or similar file transfer with FTP to see, if you achieve, better, worse or about the same performance. This could then point you in the direction to where the problem may be...

Volker.

Re: NFS transfer times

Brian Reiter — Fri, 28 Oct 2005 02:36:33 GMT

Hi Volker

FTP via other network interfaces seems OK. Certainly not as low as 0.9Mbyte.

When I'm back in the office I'll try FTP on that interface and see if there are any differences.

cheers

Brian

Re: NFS transfer times

Colin Butcher — Fri, 28 Oct 2005 03:58:45 GMT

What transfer rate do you get with other protocols (eg: DECnet COPY, TCP/IP FTP)?

I find something like COPY REMOTE:: NL: useful to remove the local system disc etc. from the equation.

Worst case write a small test program (even DCL using OPEN / WRITE / READ / CLOSE) to just hammer the network itself.

What are you seeing with the LAN device counters (LANCP / SDA will help you here). It is possible that you have a duplex mismatch, even though you think that both ends have auto-negotiated to 100FDX.

Is the switch a managed switch and does it tell you the speed / duplex state?

What firmware have you got in the DS10?

It's a steady process of elimination - prove everything else, then you can really point the finger at NFS.

Good luck. Hope this helps.

Cheers, Colin.

Re: NFS transfer times

Wim Van den Wyngaert — Mon, 31 Oct 2005 09:22:59 GMT

How did you measure the performance ?

I did a copy of a file : 20 seconds.
I did a backup of the same file to a save set : 90 seconds (the file is extended a lot of times).

Wim

Re: NFS transfer times

Wim Van den Wyngaert — Mon, 31 Oct 2005 09:27:57 GMT

Also usefull info :
$ ucx zero nfs
... do test ...
$ ucx show nfs

Wim
(still thinking in ucx)

Re: NFS transfer times

Michael Yu_3 — Tue, 01 Nov 2005 00:11:36 GMT

Hi Brian,

I suspect that the NFS client has duplex incorrectly negotiated.

If you are running with the latest LAN patch for V7.3-2, you can use mc lancp sho dev /int to see if the duplex is OK.

Otherwise check the counters using lancp show dev/count, if there are increasing errors like alignment errors, frame check errors or receive data length errors, the there is some problem with duplex between controller and switch port.

Thanks and regards.

Michael

Re: NFS transfer times

Brian Reiter — Tue, 01 Nov 2005 04:19:23 GMT

Hi Folks

Sorry about the delay getting back. Had time off to cover staff development days at the school :)

Anyway, barring a single CRC error there was nothing untoward in the MC LANCP SHO DEV/COUNTERS, SHO DEV/CHAR indicates full duplex at a 100 MB. This interface gets used a lot by the applications and normally a mismatched duplex results in the system not working too well.

I repeated the file transfer test using FTP on the interface, FTP is around 4 to 5 times faster in both directions. Although writing files back on the server unit seems slightly slower (3 Mbytes a second vs 4 Mbytes for the client writing the file), but still substantially faster than the observed NFS performance.

So it could be some issue with the disks on the Server. I'll do some more tests when I get through the pile of post-holiday paperwork :)

cheers

Brian

Re: NFS transfer times

Brian Reiter — Tue, 01 Nov 2005 08:04:11 GMT

Sorry should have been clearer with the FTP statements. FTP copying stuff from the client to the server (get from server and put from the client) are running at the speeds I'd expect (between 3 and 4 Mbytes).

cheers

Brian

Re: NFS transfer times

Brian Reiter — Thu, 03 Nov 2005 03:18:41 GMT

Hi Folks

Well after a PC upgrade at work which lost various bits of email and cookies, I can now at least provide some more information.

mc lancp sho dev/int
Indicates both sides of the network are at 100MB FULL.

There is nothing odd in the counters from LANCP (that I can see anyway).

From TCPIP SHO NFS, the NULLRECV count is high. Calls = 35099, Nullrecv = 27540. According to the unix docs I've seen all this means is that the NFS Server is not nusy enough (too many threads I'd guess).

FWIW the switch is unmanaged and shows as 100 Mb FULL.

The target disk is ODS-5 with highwater marking disabled.

I've attached a text file with the results from MONI IO, FCP and LOCK averaged out over a couple of runs of the copy procedure. I aplogise in advance for it being a non-VMS file.

Anything else I should be looking at? Or more to the point have I forgotten something obvious :)

cheers

Brian

Re: NFS transfer times

Michael Yu_3 — Fri, 04 Nov 2005 04:04:10 GMT

Hi Brian,

Please get the following output.

$ show logical tcpip$cfs*

$ sysconfig -q inet

$ sysconfig -q nfs

I suggest you follow the guidelines in the management manual to see if some of the above logicals and the attributes for inet and nfs should be modified to improve the nfs server performance.

Attributes like ovms_xqp_plus_enabled defaults to 0.

Thanks and regards.

Michael

Re: NFS transfer times

Wim Van den Wyngaert — Fri, 04 Nov 2005 05:39:54 GMT

nullrecv
Number of times an RPC call was not available when it was thought to be received.

This count is updated each time a nfsd process is scheduled but no work needed to be done. A high or increasing count here may signify to many nfsd processes.

Configured NFS server to big ?

Wim

Re: NFS transfer times

Brian Reiter — Fri, 04 Nov 2005 06:34:14 GMT

Wim,

Hmmmmm came to that conclusion. With only one client at the moment :) I'm seeing nullrecv within 2,000 of the total number of calls.

Think I would much rather know if the NFS Server was over utilised rather than under utilised.

Funnily enough reducing the number of threads had no impact on the nullrecv rate.

Ah well - could be worse I suppose.

cheers

Brian

Re: NFS transfer times

Brian Reiter — Fri, 04 Nov 2005 06:40:48 GMT

Michael,

Been using the "vanilla" setup initially. I've found that this is often good enough for our needs.

Current information (after tinkering is attached).

I'm going to try the same experiment on a DS15 system just to see what the results are :)

cheers

Brian