HTTP Slow Out Of VMS Through Firewall-1

[Robert Atkinson]

We have a curious problem with HTTP.

I'm in the middle of setting up a BL860 cluster on VMS 8.3-1H1. Everything works as expected, except HTTP.

Apache (CSWS) is serving up a static page of about 100K, but can take 10 minutes to transfer. When we look at the data coming over, we can see it writing incredibly slowly.

The data goes through Firewall-1, the new cluster is in a private LAN to stop any uneanted traffic from escaping.

Other IP protocols are fine. If I access the same page from within the LAN, it's also fine, so that rules out the network card/link. I've also loaded the page into IIS and accessed that through the firewall, which again is fine, so it seems to rule out problems with HTTP filtering.

We think there could be a problem related directly to Firewall-1 and the size of the packets VMS is presenting. I've got another PIX firewall that I could try, but the pass-thru module attached to the blades seems to refuse to negotiate down to 10/100, so we have to use hardware capable of gigabit.

I know this is a long shot, but I wondered if anyone else has come across anything similar, or could give me a clue where I coud start looking and tweaking parameters?

Cheers, Rob.

[marsh_1]

rob,

not much help but you're probably best off putting this in the network forum.

fwiw

[Hoff]

Seems that this traffic has exceeded the capabilities of this model of the Check Point Firewall-1 firewall.

I've seen a few firewalls crater exactly like this (including having protocol-specific speed differences), either due to the volume of data or due to the overhead of firewall-based inspections. Check the rules and settings and processing and NAT here, as a start.

Check with Check Point here first, or shop around for better bandwidth with another widget.

Ignoring the issue around setting the speed (which is generally via LANCP in OpenVMS I64) this looks to be the firewall.

[Wim Van den Wyngaert]

I know nothing but try netstat -s (ucx sho prot) on the browser side (pc in command prompt) before and after the request.

May be ICMP or other counters indicates something.

Wim

[David Jones_21]

We run into problems generally with our firewall and TCP window scaling.

[Wim Van den Wyngaert]

Rob,

Could you also define slow ?

Wim

[Cass Witkowski]

Do you have LAN_FLAG set in your SYSGEN parameters? If it is set to 64 then you have jumbo frames enabled. This is fine unless your LAN switches and such do not support them. If not then things can get very slow. You may not see issue unless you are trying to transfer more than 1KB at a time.

[Robert Atkinson]

Cass, we have LAN_FLAGS defaulted to '0' at the moment, although I will be switching Jumbo Frames on at some point.


Wilm, this is the original definition of slow from my first post :-

"Apache (CSWS) is serving up a static page of about 100K, but can take 10 minutes to transfer. When we look at the data coming over, we can see it writing incredibly slowly."


I'll give the netstat test a try as well.

Rob.

[Wim Van den Wyngaert]

Me bad reader. Sorry.

Also check "route print" (=ucx sho rout). May be a bad route is taken (traceroute on VMS, no idea how to do it on PC).

I also had once that 2 devices had the same IP address. 1 was behind the firewall but was able to get the arp request. It answered
but then the other node with the same IP answered too. This caused very slow communications (packets needed to be resend).

Wim

[Robert Gezelter]

Rob,

I suggest a first step toward diagnosing this is to get a trace of the affected connection. My preference is to use WireShark, as it can produce a dump file that can then be sent to whomever needs to view it.

I would also try a variety of experiments (all with the LAN monitoring in place) with different file lengths to see where the "shoulder" actually is.

- Bob Gezelter, http://www.rlgsc.com

[marsh_1]

hi,

duplicate address issues will also show with ping as it will alternate between finding address / not finding address. traceroute is tracert in dos on a windows box. if nat'ing is in in place here this can also cause issues with return addresses and the routes taken, but if other protocols to/from this box are ok this is less likely.

fwiw

[Wim Van den Wyngaert]

Duplicate addresses with 1 address behind a firewall (that doesn't let you communicate with the node in IP) will pass only ARP (in my case, may be that is a bad config). Thus the program will send a packet to a node, will not receive an ack because the firewall drops it, resend it until the right address was used.

Note : this is all 10 years ago and I could have forgotten some details.

Wim

[Hoff]

How are we on IP addressing when we have different speeds for different protocols for the same IP addresses through the same firewall?

Or was the "Other IP protocols are fine." statement incorrect?

[Robert Atkinson]

An update on this.

When I started using the HTTP connection first thing this morning, it was near immediate. This afternoon, it's gone back to being relatively slow (10 seconds for the page).

As I type, it's immediate again!

This leaves me with one conclusion, given that nothing is changing on the VMS hosts - Firewall-1 is struggling somewhere.

I've managed to get the PIX box online, so we have an alternative method of connection. As these servers will eventually come out from behind the firewall onto the main LAN, can't see any point in trying to hunt the problem down through Firewall-1.

I ran the netstat and SHOW PROT displays, and although I'm not an expert, I couldn't see anything obvious in them to point the finger.

I thank everyone for their input. I don't like leaving mysteries, but this one is so deep it could take months to resolve.

Thanks again, Rob.

[Jur van der Burg]

As said before, make a network trace. That may give an answer of where the problem may be in minutes.

Jur.

[Richard J Maher]

Hi Rob,

Looks like your problem is solved/avoided but I, for one, had never heard of Firewall 1 or its availability on VMS so I'm off reading now.

In case it would interest you and/or others here's some information I receive a few months ago, on tha subject, that I found interesting: -

"BTW, delivery of IPSEC also provides host-based firewall capability, which is another important feature that would also be delayed if IPSEC is further delayed."

Cheers Richard Maher

PS. I'm reading this bit first :-)
http://www.checkpoint.com/products/softwareblades/ipsec-virtual-private-network.html

[Robert Atkinson]

Richard, we run Firewall-1 on an appliance server - not VMS. Sorry to disappoint :)

Rob.

[Hoff]

I've been connecting into OpenVMS boxes from client boxes via L2TP / IPSec and PPTP for some years now, with this connectivity is usually based on the capabilities of the external firewall.

If OpenVMS itself sprouts L2TP or PPTP tunneling or an IP firewall (yes, I know about stunnel and the IPSec EAK) with TCP/IP Services, I might revisit the configuration I typically deploy. But for now, the approach I have works nicely from a variety of client boxes. I've worked with a couple of customer folks around firewalls and tunnels and such, including authentication, up through around allowing tunneling (with NAT) into OpenVMS boxes for use with Netbeans. (The Java RMI layer underneath Netbeans doesn't "like" NAT. But I digress.) This stuff can be gotten to work, but it's not as plug-and-play as any of us might like.

There are various firewall and tunnel server offerings here (from free with the use of your existing spare x86 hardware up to seriously expensive), and the appropriate box depends on factors including network and firewall bandwidth and authentication and syslog logging and required specific features or capabilities. Some folks need tunneling or IPSec or such. Here, the firewall processing and memory and bandwidth required to sling gigantic static HTML pages through the firewall box looks to be a central requirement.

[Robert Atkinson]

Unfortunately, this problem will remain a mystery for now (see my post above).

Thanks again for everyone's input.

Rob.

[Robert Atkinson]

Problem was found to be with port negotiation between Firewall-1 and CISCO switch - Full v Half Duplex I think.

Rob.