Cause of ICMP source_quench?

[Michael Simone]

My N4000 (11.0) server began returning source quench messages to my Windows-based network monitors (and other PCs) last night (pings were fine from Unix machines). Nothing strange had occurred last night. The machine was last rebooted 3 weeks ago (after applying our latest patchset).

The only incidents have been some recurring LAN "blips", where we briefly lose connectivity on both LAN ports. This is occurring on all our Unix hosts (7), and is likely due to our 3Com LAN equipment. But each time, the hosts have recovered within 15 seconds, and we have not experienced any continued connectivity issues.

Based on the posts here, I used the command ndd -set /dev/ip ip_send_source_quench 0 to disable the source quench messages, and I'm now able to ping without a problem from all Windows PCs.

My question is, is there any information as to why these "errors" begin? With no coinciding incident on my system, why would it suddenly begin returning these messages? Is this buffer overload condition a cumulative thing that would climax after 3 weeks of pinging?

Is there any definitive information as to whether this condition affects performance? And by disabling the messages, am I blinding myself to a potential problem, or just eliminating a useless nag.

Any information would be appreciated.

Thanks,
Mike

[Rita C Workman]

Take a look here....

http://forums.itrc.hp.com/cm/QuestionAnswer/1,1150,0xea6837f45ef7d4118fef0090279cd0f9,00.html

Regards,
Rita

[John Payne_2]

I hope you were able to get to the link Rita provided. If not, it is caused by an IP socket buffer filling, which initiates the source quench. Turning it off allows the server to ignore the fact that the buffer is full and not worry about stopping the pings or other ICMP's coming in. I think they added that to 11.X from the 'communist in every tree' theory (too worried about security to let the machine run correctly), where the machine will quench a potential attack on the server through ICMPs...

[Sridhar Bhaskarla]

By default I keep the source quenching turned off on my servers. This is becuase I have got some applications that do kind of initial error checking through the basic ICMP tests. And they fail starting up on 11.0 due to the default setting of sending source quench messages.

HP says to ignore these messages through they are indications of socket buffer full conditions. However, what I observed is that these messages are even generated from a system that is 100% idle. So, I have been taking it granted that the socket buffer full condition is not necessarily due to the system being overloaded with network activity.

You can disable it by using ndd or by changing /etc/rc.config.d/nddconf by setting ip_send_source_quench parameter to 0.

-Sri

[Anthony deRito]

Michael, as a systems administrator, it is important to understand what these events mean. Source Quench messages are the product of TCP flow control. It is true that the ICMP protocol is responsible for generating and delivering the messages but the more important thing is source quench messages are an important part of TCP flow control. I would not disable this feature... its there for a reason. These messages are used by the recieving devices to help prevent thier buffers from overflowing. The receiving devices will send a source quench message to request that the source reduce its current rate of data transmission.

The TCP buffers are intended to store occasional bursts of excess data in memory until they can be processed. THe key word here is "occasional". The buffers will overflow when data continues at high speeds and will have a detrimental effect of network performance.

You are correct thinking that you are simply "covering up" a problem. The fun is to find out why this is happening and the education you will have as a result of it will be priceless.

Happy hunting!

Tony

[James Beamish-White]

ICMP source quench can be also used as a DOS attack... so I always turn it off for internet hosts.

As an aside, I note that it is suggested to be turned off in the Bastion Host document - http://people.hp.se/stevesk/bastion11.html.

Cheers,
James

[Brad Andriese]

Michael,

Strange. I experienced the exact same problem tonight on a L2000 (11.0)as what you described. Does anyone know what causes this problem? I've never seen this before and we have not changed anything. I could find no other events on the system and no users where logged in at the time.

We have What's Up Gold that we use for monitoring our network which fired off emails to our pagers because of the "source quench" responce from ICMP pings. I tried the command "ndd -set /dev/ip ip_send_source_quench 0" and everything is back too normal.

If anyone has any ideas about this, I'd be very curious to know why this problem poped up all of the sudden.

Thanks,

-Brad

[Brad Andriese]

Sorry. Goofy me. I didn't see the other posts.