Announcement: New Performance White Paper on docs.hp.com

[Dave Olker]

A new performance white paper is now available for download from the following location:

http://docs.hp.com/en/10388/DesigningaHighPerformanceNFSServer.pdf

Here is the Executive Summary of the paper:

During a recent customer sales engagement, HP was asked to deliver an Integrity Superdome system capable of serving huge amounts of file data to the customer’s production environment. The customer’s application runs simultaneously on thousands of compute nodes. The compute nodes use the Network File System (NFS) protocol running over a TCP/IP network to retrieve application data from a central file server. These compute nodes require a sustained throughput rate of over 3 Gigabytes of file data per second.

After several rounds of benchmarking and tuning, HP delivered a 32-core Integrity Superdome system capable of serving application data at a rate of over 3 Gigabytes per second to thousands of NFS/TCP clients. This whitepaper describes the customer’s application requirements, the areas the benchmark team focused on to achieve the desired results and the final configuration of the Integrity Superdome system used to meet the customer’s throughput requirements.

While this paper discusses a specific customer engagement involving an Integrity Superdome system used as an NFS file server, most of the improvements made during the course of this effort were not specific to NFS, nor were they specific to the Integrity Superdome hardware platform. Many of the performance improvements made as a result of this engagement could potentially benefit almost any application running on nearly any system running HP-UX 11i v2.

[A. Clay Stephenson]

Hi Dave,

Those are impressive metrics. One of the tunings that jumped out at me was the 512GiB of memory with 90% dedicated to buffer cache so that once the all application data were read once, all reads would then be logical reads. I knew the buffer cache searches were greatly improved at 11.23 and up but that is a very impressive statistic.

[Dave Olker]

Hi Clay,

Thanks for reading the paper and giving your impressions. As the senior ranking ITRC Wonk, your feedback is greatly appreciated.

Dave

[Patrick Wallek]

Was this a project you were involved in, Dave? I would imagine that it was quite interesting. 3GB/sec for NFS traffic is tremendous.

What a configuration for the server. I share Clay's feeling regards the 512 GiB RAM and 90% Buffer Cache. Can you tell us what values were set for dbc_min_pct, dbc_max_pct (and any others that might effect buffer cache)? Were they set the same so that BC was essentially static?

Just curious.

[Dave Olker]

Hi Patrick,

Yes, I spent over a year working on this project and several more months documenting the results of the work we did and working with the various labs to get the performance fixes released as GR patches for HP customers.

The project itself was very interesting and at times incredibly frustrating. Trying to coordinate the efforts of 7-8 different labs, both internal and external to HP, was a bit tricky at times.

The reason for the huge memory and buffer cache footprint was specifically to get the customer's data set resident in cache to avoid any involvement from their back end storage. Both dbc_min_pct and dbc_max_pct were set to 90%, which effectively gave us a fixed sized cache. We could have just as easily left dbc_min_pct to the default 5 but I felt it was better to get the system to allocate the entire cache in contiguous memory at system boot time rather than grow the cache over time and hope for as many contiguous chunks as possible.

Thanks for your interest. Please let me know if you have any other questions or feedback on the paper.

Regards,

Dave

[Geoff Wild]

Can you explain what you need to do to accomplish this in 11.31? As there is no dbc_min_pct and max to tune.

Rgds...Geoff

[Dave Olker]

Hi Geoff,

Yes, the dbc_min/max_pct variables are not in 11.31 because we've introduce a Unified File Cache in 11.31. The variables used to size the file cache on 11.31 are:

filecache_min
filecache_max

These roughly equate to the dbc_min/max parameters as it relates to buffer cache on pre-11.31 systems. For more information on these specific variables you can refer to the filecache_min(5) man page on any 11.31 system.

Regards,

Dave

[Dave Olker]

- bump -

Man, I wish this forum supported pegged threads. Letting people know about this new technical paper.

[Alzhy]

Dave,

Kongtratz! Amazing setup indeed.

Exactly the sort of a white paper I've been loking at in a proposed re-implementation of a client's mega-application that will separate applications from databases. I'm looking at having Database "farm" environments large enough and with the needed network bandwidth to serve the application servers.

So CSO, TSO plus APA and tweaks to certain kernel params plus proper cache configuration completes the "recipe".

BTW, on the switch end - I suppose there is also a need to concentrate all the host ports on a single network blade?

[Geoff Wild]

Hey Dave - I'm looking for a 11.31 NFS performnance whitepaper.

We are now running our SAP on 11.31 Itanium, and you know that SAP uses a ton of NFS...

I'm currently trying to resolve an intermittent issue with TemSe - where an ABAP job fails because it can't close a file on a NFS mount...

At this point, I don't know if it is NFS or SAP...

I made no NFS specific tuning changes on the 11.31 system...

Rgds...Geoff