RHCS & iptables, can they live together

[Steven E. Protter]

I have an almost ready for production cluster.

It works great, has four complex services that fail over very, very nicely under all possible scenarios.

The I introduce iptables and it stops working.

I've got the private heartbeat lan, a reverse cable between to nics's wide open on all ports.

I have the 192.168.0.0 lan wide open on all ports in iptables.

The default on input output and forward is to drop.

I've added port 694 to the list of ports to permit.

My cluster sits behind a hardware firewal that is battle tested and does not permit any access on non-configured port. Its a web cluster so port 80, 443 and a few others are open on the hardware firewall.

I'd like to be able to add users that wish to abuse port 80, not to the hardware firewall which has limited configuration capabilities, but to ipables.

Two lines of defense are always better than one line of defense. Right?

RH 4 update 3 RHCS is fully up to date.

I see my options as:
1) Don't use iptables. I can go prod like this, it doesn't matter but it would be nice to make iptables play with RHCS.
2) Change the default policy to a more permissive policy. Don't know what that would be.
3) Replace the hardware firewall with something smarter and more expensive.
4) Find out exactly what ports besides 694 RHCS needs open between nodes to run and fail over properly.

The cluster has 4 floating ip's and no public ips. The firewall owns them.

I'm probably being overly paranoid, but I'd like to run iptables here.

Please let me know if I can and how.

Everyone should know my point policy is generous to itrc folks.

SEP

[Stuart Browne]

That's odd. Assuming you've got INPUT/OUTPUT chains in the firewall, with -i in INPUT, and -o in OUTPUT, it shouldn't block any traffic on those interfaces.

As it's a cross-over cable, with no possible public traffic on it, it should be just fine.

Have you watched a TCP dump output on the specific interface to ensure that the heartbeat traffic is actually going out the correct interface, and not routing out something else for whatever reason?

[Steven E. Protter]

Good idea.

Could also be as simple as leaving localhost(duh) off he configuration for my script that generates the code.

That messed a bunch of things up on a non-clustered box today.

Duh.

I'll get back to all.

SEP

[Manuel Wolfshant]

Rather then setting the default policy to DROP, I always prefer to end the rule chain with
iptables -A OUTPUT -j LOG
iptables -A INPUT -j LOG
iptables -A OUTPUT -j DROP
iptables -A INPUT -j DROP
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
This way I always know what was dropped and I can pinpoint cases where smtg is dropped but it shouldn't be.

[Steven E. Protter]

These are very good replies.

I was wrong about localhost. It was not the cause of the problem.

I'm going to take this back in my lab, since exerimenting on a couple of servers that go prod in 30 hours is not a good idea.

Going to have to do testing that is more realistic, since merely turning on iptables breaks the cluster.

As far as logging and not dropping, interesting idea, but i prefer no interaction with people using ports I have closed. I close ports for a reason. The default policy is to be a black hole. That sould not interfere with normal operations.

Good contributions, minimum 7 points both.

SEP

[Manuel Wolfshant]

You can always add specific DROP rules before the LOG ones, so that you do not end with tons of crap in the logs. And of course you can switch back to a DENY policy when you are satisfied with the configuration. However my 6yrs of experience show that a default DROP with no logs is most of the time just a source of problems. Just like yours :)

PS: do not forget that you can use -t mangle to add specific rules (either LOG or DROP/REJECT) without cluttering the main filter table and without a need to flush the main table during tests. That is, if you have no other specific need for the mangle table in your firewall.

[Bill Thorsteinson]

A couple of suggestions.

- use a switch to interconnect the cluster
members. That way they won't see a network
disconnect when another cluster member goes
down.

- use a firewall builder like shorewall.
You can easily set the policy on the interface
for the cluster connection to accept and
restrict other interfaces as needed.

[Steven E. Protter]

Shalom Bill,

I found another port the cluster was using and think I can get it to work in the lab. Hopefully at some point, the cluster services won't come down when run the iptables commands.

It might also be the way the script builds the table, which is designed to keep people from poking in while the new configuration is building.

I'll get it and thanks for the great suggestions.

SEP