11i Networking Utils

[Scott McDade]

Hello:

I suspect I am having some kind of networking issue. I have 20 - 11i systems spread across 3 subnets using NIS and several NFS mount points. Over the past 2 weeks I have been experiencing big network delays in login onto systems,PINGs take 200 - 300ms and FTPs. On top of it all I have MQ-series clients running on each 11i system and they are hanging as well.

So.... given the fact there are so many systems exhibiting the same symptoms intermittently and so many different processes using different protocols....it leads me to believe I have some kind of network performance issue. So I am hoping someone has a recommendation for a network performance tool-kit for HPUX 11i. I would like something that could store historical information such as packet loss, collision and response times. And oh yeah something that doesn't require you to be a network engineer to use or understand the data.

Can anyone help?

[Ron Kinner · Accepted solution]

The classic tool for this is MRTG. Runs on almost anything and is free. You will need to enable SNMP on either the switch or the 11i. In its default configuration MRTG just monitors the number of bytes in and out on an interface and draws you a cute graph of traffic over time. With a little tweaking it will happily graph any variable that SNMP can report which includes errors and collisions. Does help to be a network engineer to set it up but the instructions are pretty good.

http://people.ee.ethz.ch/~oetiker/webtools/mrtg/

Another freebie is getif. Only runs on a window box but it is much easier to setup and use than MRTG and can also do graphs.

http://www.wtcs.org/snmp4tpc/getif.htm

Then you have two very good builtin programs on your 11i which you should look at to help you determine where the problem is.

lanadmin
lan
display

The result tells you a lot about the traffic and errors on the network card. (If you have more than one you will need to change to a different PPA to see its traffic just do
ppa x
display)

The second page of the display has most of the errors. The number and type of errors can tell you a lot about the condition of the link. Most common problem is a duplex mismatch where one end is set to 100 FULL and the other to AUTO. This results in high collisions on one end and frame check sum errors on the other.

Another very useful tool is linkloop. Look in the man for details but essentially you just need to know the MAC of another device on your LAN. Then
linkloop -n 100 -s 1400 -v MAC
will give the switch and the two links involved a good test.

What kind of switch do you have anyway? Is there more than one switch involved? If so how are they connected? Have you checked your switches for errors?

TCPDUMP and Snort are also good but again it helps to be a network engineer if you want to use them.

Ron

[Michael Steele_2]

System performance analysis is point to point. So unless you're using a network analyzer like tcpdump, which most S.A.'s now keep in their tool box, then similar applications have to exist on both nodes. For example, spray. This is an icmp protocol. Set up spray on both nodes and then spray one node and check for dropped packets, collision, retransmissions, timeouts, etc.

nfsstat -rc (* NFS *)
netstat -m (* requests denied *)
vmstat -n (* cpu usage *)
netstat -s -p udp/tcp/icmp (* socket overflows, etc. *)

Also bear some mind to packet reassembly if you've got a lot of routers.

[Steven E. Protter]

1) Ping test, watch for variability. This could confirm what you already know.

2) traceroute

This test could let you know what part of your infrastructure is slowing you down. Not too useful for hubs, very useful for recalcitrant switches.

If you have cisco switches, see that the HP-9000's ports are manually set to 100 BaseT Full duplex, or whatever your NIC top speed/mode is.

You may need to force your lan cards to run full speed by adding speed entries to /etc/rc.config.d/hpbtlanconf

Example 2 cards.

HP_BTLAN_INTERFACE_NAME[0]=lan0
HP_BTLAN_STATION_ADDRESS[0]=
HP_BTLAN_SPEED[0]=10HD
HP_BTLAN_INTERFACE_NAME[1]=lan1
HP_BTLAN_STATION_ADDRESS[1]=
HP_BTLAN_SPEED[1]=100FD

lanadmin should when complete show 100 BaseT full duplex MANUAL

SEP

[Bill Douglass]

I would suggest running tcpdump, ethereal of even HPs nettl utility. Configure the tool to watch all packets between your server and the host you are logging in from.

Once you capture the login session, you can check for things the three-way handshake (SYN, SYN-ACK, ACK) that starts up any TCP connection, re-transmitted packets, protocol errors, etc.

You coud also get a too like netperf

http://www.netperf.org/

to generate some maximum throughput numbers between systems.