ping "source quench" issue

[TwoProc]

A while ago - we had an event where we couldn't "reach" the database server - it was practically a campus-wide problem. When I pinged the server, I got:

From xxxxxxx.xxxx.xxx (xxx.xx.xx.xx): icmp_seq=25 Source Quench
64 bytes from xxxxxxx.xxxx.xxx (xxx.xx.xx.xx): icmp_seq=28 ttl=254 time=0.162 ms

And, just like that - it was over.

What is/was a source quench?

What was quenched, my workstation that I pinged with, or the server that I pinged at ?

We've been having a few connection related issue over the past few weeks (presumably network related).

Is this typically an OS issue? Is this typically a networking issue? If networking, at the routing/switching layer?

This is really ugly when it occurs, any suggestions would be really appreciated.

Oh, another data point - about three days ago it happened as well (very similar problem), but *that* time when it happened, when I pinged the server - over 30 pings it showed me 81% percent packet loss (not a Source Quench).

Any suggestions/tips/advice?

My thanks to those who bother to post!

[James R. Ferguson]

Hi:

Source quench messages indicate that network buffer space can't handle the traffic rate.

You should be able to eliminate the message (at least) with:

# ndd -set /dev/ip ip_send_source_quench 0

Regards!

...JRF...

[TwoProc]

JRF - thanks for the reply - but I knew I could turn it off - I just would like to know why I'm getting it. In light of the fact that I've got other network connectivity issues, I believe that they are related. But, to me, turning it off isn't the issue - the issue is somehow we ????flooded??? a 4 way 1Gb fiber port aggregate. Kind of hard to believe that I just need to turn off source quench notifications - I think I've got other issues. Of course, they are intermittent, meaning they are a nightmare to figure out.

[Bill Hassell · Accepted solution]

> the issue is somehow we ????flooded??? a 4 way 1Gb fiber port aggregate.

It's not the NICs that are flooded. The driver is too slow to handle all the packets. Make sure all your HP-UX systems running multiple Gbit cards have the very latest network patches. TCP/IP is quite versatile but isn't particularly efficient. That's why jumbo frames (9000 vs. 1500) will improve throughput. Using top or Glance, when you see the source quench error, do you also see very high system (kernel) usage? This is an indication that the drivers and network subsystems are running as fast as they can.

A 4-port Gbit aggregate (when all NICs are heavily loaded) will be pushing close to 300 Mbytes/sec using jumbo frames. Unlike disk drivers that can initiate I/O that is 100 times larger than network traffic. All those tiny protocol packets will really stress the kernel drivers and subsystems.

It sounds like you have multiple problems. Be sure to run netstat -i and lanadmin -g , looking at the second section with error counters. You might start by testing server-to-server crossover cables to eliminate possible switch issues.

[TwoProc]

Thanks Bill - I do think it's NW - just due to the fact that we've seen other problems, however, the correct response for me is to review all of this that you've indicated, as well as anything that comes up.

My thanks for the response,

J