why source quench?

[zhaogui]

When I ping some hp-ux servers
I got "Source quenched" but other hp-ux servers in different subnet is ok. According to http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xa041af48a9e5d5118ff40090279cd0f9,00.html
I got 1 for all hp-ux servers after running ndd -get /dev/ip ip_send_source_quench. After changing to 0 then it is ok. May I know how come some servers need to adjust this parameter but some don't need to?

Thanks,

[Clemens van Everdingen · Accepted solution]

Hi,

Here is the answer:

RFC792 says in its page 10:

A gateway may discard internet datagrams if it does not have the buffer space needed to queue the datagrams for output to the next network on the route to the destination network. If a gateway discards a datagram, it may send a source quench message to the internet source host of the datagram. A destination host may also send a source quench message if datagrams arrive too fast to be processed. The source quench message is a request to the host to
cut back the rate at which it is sending traffic to the internet destination.

RFC1122 says:

3.2.2.3 Source Quench: RFC-792

A host MAY send a Source Quench message if it is approaching, or has reached, the point at which it is forced to discard incoming datagrams due to a shortage of reassembly buffers or other resources. See Section 2.2.3 of
[INTRO:2] for suggestions on when to send Source Quench.

[INTRO:2] is RFC1009 - Requirements for Internet Gateways - and it says:

2.2.3. Source Quench

All gateways must contain code for sending ICMP Source Quench messages when they are forced to drop IP datagrams due to congestion. Although the Source Quench mechanism is known to be an imperfect means for Internet congestion control, and research towards more effective means is in progress, Source Quench is considered to be too valuable to omit from production
gateways.

From these, you can notice Source Quench is not MUST item.
And internet hosts can choose whether or not it will be generated.

At 11.0, its implementation has dramatically changed from 10.X.
When it receives ICMP ECHO packets, they are all sent upstreams to any application which opens SOCK_RAW socket. And if some of
those program's socket buffer gets full just because the program can not read those ICMP ECHO packets in timely manner, ICMP SOURCE QUENCH is generated.

The above behaviour is specific one to 11.0 and can not be seen on 10.X.

Hope this explains.

C.

[U.SivaKumar_2]

hi,
The reason for only some servers giving you source quench while some servers works is that
it depends upon the network traffic load , the
amount of network memory buffer available at that point of time and even the difference in buffer parameters of network adapter cards in the different servers may cause source quench.
regards,
U.SivaKumar

[Clemens van Everdingen]

Hi,

Can only happen on HP-UX 11.00.

C.

[zhaogui]

How can I identify the root cause? What command can I use to check the network memory buffer size? Will that affect our daily operation? I found out that I can still telnet,ftp but not sure what's the impact on overall operation.

[Clemens van Everdingen]

Hi,


" Is there any bad side effect from this change ? "

"The answer is NO. ICMP source quench is not generated at 10.X and there is no problems reported with this behaviour. And as you
can notice in the RFCs above, ICMP souce quench is the feature of "MAY". Hosts just "may" send it."

So no worries, just use ndd to set the parameter to 0 and the problem is gone.

C.

[zhaogui]

Actually my concern is if I don't make any change will there be any bad side effect or implication on our daily operation?

[Clemens van Everdingen]

Hi,

No !!

But why leave the irritating messages if you can get rid of them ?

C.

[Ron Kinner]

I asked that same question a few months ago and someone pointed me to this article:

PROBLEM
Upon pinging an 11.0 system, I am seeing a packet loss and Internet
Control Message Protocol (ICMP) source quench messages.

Why am I getting these messages?

CONFIGURATION
Operating System - HP-UX
Version - 11.0
Subsystem - Internet Control Message Protocol (ICMP)
RESOLUTION
ICMP source quench messages are generated when an IP packet is
received by the 11.0 system that can't be delivered to the socket
buffer of the receiving application. The intent is to inform the
sender of the full buffer condition so the rate of the transmission
is slowed down until the buffer can be read by the receiving
application.

Setting the ndd parameter ip_send_source_quench to 0 can be an
effective way to deal with the messages.

Programs that use icmp protocol, such as ping, use a type of
socket called SOCK_RAW. The nature of using raw IP sockets is
that ALL packets received that match the protocol type of the raw
socket are delivered to ALL the sockets using that protocol. It is
up to the application to read all the data in it's socket buffer
and discard the data it's not interested in. If any of these
sockets are full, the icmp source quench message will be generated.

One process that uses one of these sockets is part of DCE, and it
is 'rpcd'. This program opens a raw socket in order to listen for
icmp messages, which it uses to monitor the health of other systems
on the network running DCE. In this case 'rpcd' used a 32K buffer,
and processed the messages received every 5 minutes, which led to
the buffer full condition.

PHSS_17810 addresses the problem by increasing
the buffer size to 128K and processing the messages every 2 minutes.

Sorry I've lost the URL.

So there is a patch which might correct the problem but most people just turn it off which is what we did. No bad effects that we can see in several months of operation.

Ron

[MANOJ SRIVASTAVA]

Hi Zhaogui ,

In HPUX 11.00 there is a patch to avoid this problem. Here is gist of HP's reply:

This problem will be fixed in the 11.01 version of the HP-UX operating
system. These messages can be safely ignored as they have absolutely no
impact on the operating system (performance or otherwise). Alternatively
these messages can be prevented by disabling source quench. For more
information see the sections below.

What is causing these messages?

At 11.0 the Streams Xport layer now passes the ICMP echo request to any
other process that has a socket open and bound to raw IP. The rpcd
rpcd/dced deamon opens a raw socket to listen to ICMP messages. This raw
socket is open by icmp_monitor routine of rpcd. The main function of
this routine is to check for error messages from DCE servers registered
in endpoint database of the host and it checks the socket every 5
minutes. It does not respond to or use the ICMP echo requests; however,
the socket queue becomes filled during the 5-minute delay causing the
source quench message. The fix being implemented in 11.01 will be to
increase the buffer size to 128 K and shorten the wait interval from 5
minutes to 2 minutes, thereby flushing the queue of these unwanted
messages before the queue becomes filled.

Why is it safe to ignore these messages or to turn them off?

A good disscussion of this is in TCPIP Illustrated Volume 1,
by Richard Stevens, pages 160-162.Here is an excerpt from page 161:

Although RFC 1009 [Braden and Postal 1987] requires a
router to generate source quenches when it runs out of
buffers, the new router requirements RFC [Almquist 1993]
changes this and says that a router must not originate
source quench errors. The current feeling is to deprecate
the source quench error, since it consumes network bandwidth
and is an ineffective and unfair fix for congestion.

Also, see RFC 1812, section 4.3.3.3 Source Quench for a good
discussion of the issues.

Also RFC 2001 has discussion on Congestion and why TCP should handle
this not ICMP:

Congestion avoidance and slow start are independent
algorithms with different objectives. But when congestion
occurs, TCP must slow down its transmission rate of packets
into the network and then invoke slow start to get things
going again. In practice, they are implemented together.

Exactly how do I disable source quench?

You can disable source quench in HP-UX 11.0 by executing this command:

ndd -set /dev/ip ip_send_source_quench 0

To disable Source Quench so that it can survive a reboot,

modify the /etc/rc.config.d/nddconf file as follows :
TRANSPORT_NAME[X]=ip
NDD_NAME[X]=ip_send_source_quench
NDD_VALUE[X]=0

Where X is the next logical numerical sequence in a table
of values, with X starting at 0.
--------------------------------------------------

DCE patch and dependencies:

3 PHCO_23651 fsck_vxfs(1M) cumulative patch
2 PHCO_23876 cumulative SAM/ObAM patch
1 PHCO_25883 cumulative 10.20 libc compatibility support
3 PHKL_18543 PM/VM/UFS/async/scsi/io/DMAPI/JFS/perf patch
3 PHKL_20016 2nd CPU not recognized in G70/H70/I70
3 PHKL_23956 Profile, virtual timers and disabling fix
3 PHKL_24027 VxFS 3.1 cumulative patch
1 PHKL_25906 Probe,IDDS,PM,VM,PA-8700,asyncio,T600,FS
1 PHKL_25999 syscall, msem_lock, umask cumulative patch
2 PHSS_21614 HP DCE/9000 1.7 Runtime cumulative patch



Manoj Srivastava