Hp ProCurve 5406zl truncates the ip packets size on mirror port.

[ergin_ozekes]

Hi All;

I am trying the packet forwarding to iptables from mirror port. I've installed the port mirroring on Hp ProCurve 5406zl via below cli scripts:

ip access-list extended "mirror-acl"
10 permit tcp any any eq 80
20 permit tcp any any eq 443
30 permit tcp any any eq 8080
40 deny ip any any
exit
mirror 1 port E2
interface E24 monitor ip access-group "mirror-acl" in mirror 1
write memory

E2 is the mirror port and the E24 is monitor port from which the frames copied to E2 port.

E2 port has been connected to packet analyzer server's eth0 port which is in promiscuoues mode. I've installed the bridge from eth0 to tap0 by using below scripts:

modprobe tun
tunctl -u root

ifconfig tap0 0.0.0.0 promisc up
ifconfig eth0 0.0.0.0 promisc up

brctl addbr br0
brctl addif br0 eth0
brctl addif br0 tap0
brctl stp br0 off
ifconfig br0 up
dhclient br0

echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward

After this setup I can see frames for web services on the eth0 via "tcpdump -i eth0 -n port 80" as below:

17:09:19.734166 IP 208.117.254.21.80 > 10.237.11.177.41399: Flags [S.], seq 2610898202, ack 3807573563, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
17:09:19.744377 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [.], seq 8871:10331, ack 1, win 594, length 1460
17:09:19.744393 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [.], seq 10331:11791, ack 1, win 594, length 1460
17:09:19.744401 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [.], seq 11791:13251, ack 1, win 594, length 1460
17:09:19.744410 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [P.], seq 13251:14591, ack 1, win 594, length 1340
17:09:19.797444 IP 208.117.254.21.80 > 10.237.11.177.41400: Flags [.], ack 370, win 473, length 0
17:09:19.797597 IP 208.117.254.21.80 > 10.237.11.177.41399: Flags [.], ack 367, win 473, length 0
17:09:19.797767 IP 208.117.254.21.80 > 10.237.11.177.41400: Flags [P.], seq 1:580, ack 370, win 473, length 579
17:09:19.797781 IP 208.117.254.21.80 > 10.237.11.177.41399: Flags [P.], seq 1:141, ack 367, win 473, length 140
17:09:19.802844 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [.], seq 14591:16051, ack 1, win 594, length 1460
17:09:19.802859 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [P.], seq 16051:17451, ack 1, win 594, length 1400
17:09:19.977466 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [.], seq 17451:18911, ack 1, win 594, length 1460
17:09:19.977480 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [.], seq 18911:20371, ack 1, win 594, length 1460

As you can be seen the maximum http packet length was 1460 size in bytes. But the packet size on the originating client of this web traffic was different as below:

17:10:35.858989 IP 64.15.117.185.80 > 10.237.11.177.49034: Flags [.], ack 1, win 473, length 0
17:10:36.942766 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [P.], seq 1:3152, ack 1447, win 594, length 3151
17:10:36.942870 IP 10.237.11.177.58174 > 64.15.117.151.80: Flags [.], ack 3152, win 286, length 0
17:10:37.118100 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [.], seq 3152:6072, ack 1447, win 594, length 2920
17:10:37.118173 IP 10.237.11.177.58174 > 64.15.117.151.80: Flags [.], ack 6072, win 332, length 0
17:10:37.118211 IP 64.15.117.151.80 > 10.237.11.177.58174: Flags [P.], seq 6072:8872, ack 1447, win 594, length 2800
17:10:37.118233 IP 10.237.11.177.58174 > 64.15.117.151.80: Flags [.], ack 8872, win 335, length 0
17:10:37.219928 IP 10.237.11.177.41399 > 208.117.254.21.80: Flags [S], seq 3807573562, win 29200, options [mss 1460,sackOK,TS val 29610864 ecr 0,nop,wscale 7], length 0

The maximum packet size was greater then the mirror port's packet's maximum size.

I can collect the frame by using below ebtables command.

 #> ebtables -t broute -A BROUTING -i eth0 -p IPv4 -j redirect --redirect-target DROP --log-prefix "EBT-BRT-ALL-LOG"

and the below command can show the increasing pcount and bcount values

#>ebtables -t broute -L --Lc

Bridge table: broute

Bridge chain: BROUTING, entries: 3, policy: ACCEPT
-p IPv4 -i eth0 --log-level notice --log-prefix "EBT-BRT-ALL-LOG" -j redirect --redirect-target DROP, pcnt = 37690 -- bcnt = 3334893

but below ebtables commands can not 

#>ebtables -t broute -A BROUTING -i eth0 -p IPv4 --ip-proto tcp --ip-sport 80 -j redirect --redirect-target DROP
#>ebtables -t broute -A BROUTING -i eth0 -p IPv4 --ip-proto tcp --ip-dport 80 -j redirect --redirect-target DROP

#>ebtables -t broute -L --Lc

Bridge table: broute

Bridge chain: BROUTING, entries: 3, policy: ACCEPT

-p IPv4 -i eth0 --ip-proto tcp --ip-sport 80 -j redirect --redirect-target DROP, pcnt = 0 -- bcnt = 0
-p IPv4 -i eth0 --ip-proto tcp --ip-dport 80 -j redirect --redirect-target DROP, pcnt = 0 -- bcnt = 0

Do anyone has idea to fix this problem?

Best Regards,

Ergin Ozekes

[Richard Brodie_1]

It looks very much like you are getting bogus results from the web client captures because you have TCP offloading enabled.

http://wiki.wireshark.org/CaptureSetup/Offloading