ProLiant Servers (ML,DL,SL)
cancel
Showing results for 
Search instead for 
Did you mean: 

ilo2 (2.15) PMTU discovery broken

 
Ruben_Herold
Advisor

ilo2 (2.15) PMTU discovery broken

hi,

 

I was trying to connect to some ilo card via ipsec vpn. The connections are very slow, so I did an tcpdump on the gateway to the ilo card and it seems that the ilo2 ignores pmtu discovery:

 

20:27:49.899451 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:49.899489 IP 192.168.101.65 > 192.168.101.112: ICMP 192.168.56.93 unreachable - need to frag (mtu 1430), length 556
20:27:49.922517 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:49.922553 IP 192.168.101.65 > 192.168.101.112: ICMP 192.168.56.93 unreachable - need to frag (mtu 1430), length 556
20:27:49.927440 IP 192.168.56.93.50366 > 192.168.101.112.443: tcp 0
20:27:49.933959 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:49.933990 IP 192.168.101.65 > 192.168.101.112: ICMP 192.168.56.93 unreachable - need to frag (mtu 1430), length 556
20:27:49.945002 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:49.945032 IP 192.168.101.65 > 192.168.101.112: ICMP 192.168.56.93 unreachable - need to frag (mtu 1430), length 556
20:27:49.955965 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:49.955998 IP 192.168.101.65 > 192.168.101.112: ICMP 192.168.56.93 unreachable - need to frag (mtu 1430), length 556
20:27:50.956805 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:50.958898 IP 192.168.56.93.50366 > 192.168.101.112.443: tcp 0
20:27:50.959495 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:50.961385 IP 192.168.56.93.50366 > 192.168.101.112.443: tcp 0
20:27:50.962036 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:50.963814 IP 192.168.56.93.50366 > 192.168.101.112.443: tcp 0
20:27:50.964363 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460
20:27:50.966148 IP 192.168.56.93.50366 > 192.168.101.112.443: tcp 0
20:27:50.976079 IP 192.168.101.112.443 > 192.168.56.93.50366: tcp 1460

 

Can I manual lower the mtu on the ilo? Cause this behavior is complete broken and violates RFC 1191.

 

 

 

 

11 REPLIES
Oscar A. Perez
Honored Contributor

Re: ilo2 (2.15) PMTU discovery broken

iLO2 TCP/IP stack is defined as a Host, not a Relay. The TCP/IP standard requires relaying to be Off as default.




__________________________________________________
If you feel this was helpful please click the KUDOS! thumb below!
Ruben_Herold
Advisor

Re: ilo2 (2.15) PMTU discovery broken

 

This has nothing to to with Host, Router, Relay and so on. PMTU discovery should to be activ on all devices connected to the internet, or must be reachable via links with an smaller mtu than 1500. Please take a look to:

 

http://en.wikipedia.org/wiki/Path_MTU_Discovery

 

"For IPv4 packets, Path MTU Discovery works by setting the Don't Fragment (DF) option bit in the IP headers of outgoing packets. Then, any device along the path whose MTU is smaller than the packet will drop it, and send back an Internet Control Message Protocol (ICMP) Fragmentation Needed (Type 3, Code 4) message containing its MTU, allowing the source host to reduce its Path MTU appropriately. The process is repeated until the MTU is small enough to traverse the entire path without fragmentation."

 

 

Oscar A. Perez
Honored Contributor

Re: ilo2 (2.15) PMTU discovery broken

Sorry, I misread your post.

 

Does this issue happen with iLO2 v2.09?  

 

I'm asking because in version 2.12 we added few countermeasures for ICMP blind reset attacks. In the ICMP unreachable message that we get back, we now check the Internet Header + the 64 bits of Data Datagram portion. If it doesn't match with the IP message that we send, we discard the ICMP.

 




__________________________________________________
If you feel this was helpful please click the KUDOS! thumb below!
Ruben_Herold
Advisor

Re: ilo2 (2.15) PMTU discovery broken

With 2.09 it seems too run fine. I  could do some more network dumps the next day.

If you send me your e-mail Address  I can send you the pcap file with the problem in 2.15.

 

Oscar A. Perez
Honored Contributor

Re: ilo2 (2.15) PMTU discovery broken

I sent you a PM




__________________________________________________
If you feel this was helpful please click the KUDOS! thumb below!
Oscar A. Perez
Honored Contributor

Re: ilo2 (2.15) PMTU discovery broken

We found the bug that broke Path MTU discovery in 2.12 and later and we are fixing it in the next iLO2 v2.20 that will be available on the FTP on early May 2013.




__________________________________________________
If you feel this was helpful please click the KUDOS! thumb below!
Ivan_Kuznetsov
Occasional Contributor

Re: ilo2 (2.15) PMTU discovery broken

Hello!

 

It seems that PMTU discovery problem exists in iLO4 firmware too.

 

We have a customer far from us with two brand new DL360p Gen8 servers we should set up remotely. The servers has iLO4 Advanced I can reach through an IPsec VPN tunnel. I can ping iLO and ssh to it but can't reach the web interface. Tcp connection to https service is established but freezes quickly. I take tcpdump and see that some "big" packets (~1400 bytes) from iLO do not reach browser. We still investigate the problem but it looks similar the discussed above

 

iLO4 firmware version is 2.13

Oscar A. Perez
Honored Contributor

Re: ilo2 (2.15) PMTU discovery broken

Please PM me a network trace if you can.




__________________________________________________
If you feel this was helpful please click the KUDOS! thumb below!
TD-NorNet
Occasional Contributor

Re: ilo2 (2.15) PMTU discovery broken

The problem also occurs in my setup, with ilo2 firmware 2.15.

 

I have captured a pcap packet trace (stored here: http://www.exp-math.uni-essen.de/~dreibh/temp/ilo.pcap) on a router connecting a local Ethernet (MTU: 1500) with a VPN tunnel (MTU: 1472). The iLO machine is 10.1.2.241, the web browser is 10.1.1.50. When the iLO machine sends a 1500-byte packet, with Don't Fragment flag set in the IP header, the router correctly responds with an ICMP Fragmentation Needed. However, the iLO machine just ignoes that and keeps sending 1500-byte-packets as retransmissions -- which are dropped. The HTTPS connection therefore hangs.

 

When will there be a fix for this problem? I think trying to use iLO over a VPN is not that uncommon.

TD-NorNet
Occasional Contributor

Re: ilo2 (2.15) PMTU discovery broken

I can also reproduce the problem with Telnet instead of HTTPS. Login from the remote side works (since the Telnet packets are small enough for the tunnel), but e.g. the response to "help" causes the iLO machine to send a 1500-byte packet -- i.e. it will not pass the router and the iLO machine does not handle the ICMP Fragmentation Needed.

Oscar A. Perez
Honored Contributor

Re: ilo2 (2.15) PMTU discovery broken

The path MTU is fixed in version 2.20
http://h30499.www3.hp.com/t5/ITRC-Remote-Lights-Out-Mgmt-iLO/iLO-2-Firmware-version-2-20-released/td-p/6067313



__________________________________________________
If you feel this was helpful please click the KUDOS! thumb below!