To PMTU or not to PMTU

[Ralph Grothe]

Hi,

although the underlying question relates to a Solaris box I think the topic is a general one applying to most Unices (hm anyway, possibly matter of taste or at least controversially debated?).

Some of this server's clients sit behind a firewall that I would consider "malconfigured" (or at least not RFC1191 conforming).
I'm not sure whether it's the FW admins' paranoia for any ICMP DoS attack or simply inferior HW that lead them to suppress all DF bit induced "Destination Unreachable" ICMP replies.

Were it on an HP-UX box then probably strategy 2 of the TCP/IP stack via the ip_mtu_strategy parameter could be selected.
Unfortunately the Solaris TCP/IP stack isn't that sophisticated in that respect.

http://www.sean.de/Solaris/soltune.html#pathmtu

But the general recommendadion there seems to be to cling to RFC and leave path mtu discovery enabled.

The network admin of this LAN wants me to remove the DF bit from outgoing large PDUs.
Since apart from this weird site there are hundreds of other clients from other (firewalled) LANs that don't encounter any problems, I'm pretty unwilling to give in because I fear to waste resources for all other clients that could no longer exploit the full frame size.

As a compromise I suggested to reduce the MTU for the IP address that those sites' clients connect to (n.b. the NIC is multihomed with IP addresses for seperate DB instances that service different sites).
This would, I hope, only affect a few clients rather than the majority.

Should I try to evaluate the path MTU myself by iterating sending packets of increasing size until they don't pass the hop,
or are there general best practices/assumptions for establishing the path MTU?

In the online help of HP-UX's ip_pmtu_strategy
for instance they talk about a table of "popular media MTUs":

If the gateway does not provide next hop information, then IP
will reduce the MTU to the next lower value taken from a table
of "popular" media MTUs.

Are these vendor specific, or could they be found in the RFCs?

What are your views on this not so uncommon nuisance of "black hole routers", as I read somewhere else?
I do need to arm myself with some arguments to support my intuitive laymen's postion.

Regards
Ralph

[Tim D Fulford]

Hi

I think the whole crux of what you are trying to say is that you have tuned your system(s) as you like them, and someone else is basically disagreeing with you.

The technical arguments should boil down to one of 5 options
1 - making the changes will break things 100%
2 - making the changes will degrade performance/things
3 - no NOTICABLE difference
4 - making changes will improve performance/things
5 - making changes will fix things 100%

From what you are saying I think you are at option 3. Leaving the MTU at 1500 or wjatever the default is & letting the F/W or LAN guys have their way will probably not effect your users.. if it does break things.. well you are perfectly placed to say "told you so".

That said... netperf (www.netperf.org) is a tool you could use to generate packets and push the networks to their limits..

Regards

Tim

[rick jones · Accepted solution]

Sun/Solaris needs all the cycles it can spare, so if you can keep the PMTU then that would be good.

Trying to evaluate the MTU to specific destinations is problematic for Internet destinations as the PMTU can change over time.

The gateway not providing PMTU information still presumes (IIRC) that there is an ICMP Destination Unreachable message that does reach the host. Otherwise, the host is left to wait for a retransmission timeout. The "table" probably does have suggested values in an RFC - I'd start with RFC1191 if memory serves.

Personally, I think these blackhole routers should be taken-out into the street and crushed. I'm undecided about their administrators :) One would think that say in the hosts a lower bound could be set to say 576 bytes, and if an ICMP Destination Unreachable, Datagram Too Big message came-in with a value smaller than that then one simply disables PMTU for that destination and uses 576 bytes. That way, when there is no attack, one is no worse off than with PMTU dissabled across the board and if there is not, one is also no worse off than with PTMU disabled.

At the crux of the matter is being able to "trust" the ICMP message came from an actual router. That would likey require digitial signatures and dollars to doughnuts the router guys do not really want that overhead.

You may end-up having to disable PMTU in the end anyway and live with the increase in CPU util on your server.

[rick jones]

Sun/Solaris needs all the cycles it can spare, so if you can keep the PMTU then that would be good.

Trying to evaluate the MTU to specific destinations is problematic for Internet destinations as the PMTU can change over time.

The gateway not providing PMTU information still presumes (IIRC) that there is an ICMP Destination Unreachable message that does reach the host. Otherwise, the host is left to wait for a retransmission timeout. The "table" probably does have suggested values in an RFC - I'd start with RFC1191 if memory serves.

Personally, I think these blackhole routers should be taken-out into the street and crushed. I'm undecided about their administrators :) One would think that say in the hosts a lower bound could be set to say 576 bytes, and if an ICMP Destination Unreachable, Datagram Too Big message came-in with a value smaller than that then one simply disables PMTU for that destination and uses 576 bytes. That way, when there is no attack, one is no worse off than with PMTU dissabled across the board and if there is not, one is also no worse off than with PTMU disabled.

At the crux of the matter is being able to "trust" the ICMP message came from an actual router. That would likey require digitial signatures and dollars to doughnuts the router guys do not really want that overhead.

You may end-up having to disable PMTU in the end anyway and live with the increase in CPU util on your server. You can check your average packet sizes on outbound by grubbing through netstat -s statistics. I'd suggest taking them over an interval.

And if you do run-out of CPU, I'm sure we could find a suitable HP server to replace your Sun system :)