ignite-ux and multiple firewalls

[Team Leader UNIX DBA]

Hi all,

Here is an interesting situation, I am wondering what advice folk would have for me.

I work at a managed services firm, and we have 11 hp-ux customers (with a total of about 25 servers), each customem is behind a firewall so that they are seperate from each other. I am wanting to setup ignite-ux in a central location to create recovery archives of these clients. They already make use of a central Data-Protector cell manager, but I want to extend the recovery policy to include Ignite-UX.

My problem is that I do not know what is required to get Ignite-UX working through multiple firewalls...

Since there are 11 client networks, does this mean that I would need 11 network cards, each connected to one of the client networks, for Ignite to work?

Or is it possible to create 11 virtual network interfaces on the one network card, each with an IP address from a client network, and then configure the switches to patch that link into each of the 11 client networks?

Or can I configure the 11 firewalls to allow Ignite network traffic through? This might be hard to do since ignite uses a lot of insecure protocols (such as remsh, nfs etc). Also, I don't even know if this would work because when recovering from a failure, how could we get the client server to boot from the ignite-ux server?

It might even be more feasable to purchase 25 DDS3 tape drives and attach then to each server and run ignite to the local tape device?

Does anyone have any ideas or recommendations given this scenario? It seems difficult to achieve this. Ideas?

Thanks all
Andrew

[Matti_Kurkela · Accepted solution]

Any method of centralized Ignite backups will by necessity weaken the separation between the clients. The question is, can you arrange it so that the separation remains "strong enough" to satisfy your clients' needs and contractual requirements?

The virtual network interfaces should be a technically feasible solution, but remember that the physical network interface that contains the virtual interfaces will get 11 networks' worth of broadcast traffic. This might degrade its performance somewhat. It will also introduce a single vulnerable point: cracking that one server will allow very powerful access to all your clients' networks. "Technically possible but not recommended."

To allow Ignite through firewalls, you would have to make the NFS component of the Ignite service more firewall-friendly. For HP-UX 11.11, a patch PHNE_35418 (or a superseding patch) will add a feature to the NFS service: it allows you to limit the number of ports NFS uses. After this fix has been implemented, NFS will require only four fixed ports, instead of a large range.

HP-UX 11.31 might already have this feature as standard, but I'm not sure about 11.23.

You would also have to install an "Ignite-UX boot helper" to each client's network. It's a small piece of software that forwards the boot query broadcast to the Ignite-UX server and supplies the initial response back to the booting client.

Modern versions of Ignite-UX can apparently be configured to use SSH instead of remsh. I haven't had the chance to verify this in action yet, but it sounds good.

To sum up: I would first check if this is allowed by contracts/regulations/whatever, then would prefer to go the firewall route if possible. It will be fairly complex, but should still be worth it.

The local tape drives are a sure solution, but remember that you'll need the manpower to exchange the tapes once in a while. Also think about major disasters: if a fire destroys a server, will the Ignite backup of that server be lost too?

MK

[VK2COT]

Hello Andrew,

I will give you my private opinion. I worked
for many years in Unix support (23 actually).
That includes support at Universities,
banks, Fortune-500 companies, hospitals,
government bodies, defense organizations,
ISPs (I managed and co-owned a Linux-based ISP
for 10 years in Australia), and so on.

a) Under no circumstances let the traffic
of different customers end on the same server.

Ensure full physical and logical isolation
of each customer.

In our business, the proper logic that each
support team shoudl ask is:

When we get hacked, how to minimize the loss?

It is not "If" but "When"... Anything
that is connected to network is AT RISK!

I practice martial arts (Shotokan-style karate) and one of my trainers used to say:

If your hand gets chopped off, you grab it in
your other hand and keep on attacking the
bad guys :)

Defend and protect your customers like you do
your own home.

b) In proper IT management (especially
outsourcing), having a happy customer
(even at a bit higher cost of support)
is CHEAPER and better than looking for a new one.

I know this because I worked on many projects
for, and with, CSC, Fujutsu, EDS, IBM GSA,
and some other smaller outsourcers in the
past.

Of course, none of them like to talk about lost customers and millions spent on new
bids because an unhappy customer walked away.
They all broadcast when they grab a new
customer, but you seldom see an announcement
about lost customers.

c) Many firewall rules are needed for
proper operation of Ignite.

I enclose herewith the HP document
that talks about Bastille and Ignite:

pass in log quick proto udp from any to any port = 69 keep state
pass in log quick proto udp from any port = 68 to any port = 67 keep state
pass in log quick proto tcp/udp from any to any port = 111 keep state
pass in log quick proto tcp from any to any port = 135 keep state
pass in log quick proto udp from any port = 1068 to any port = 1067 keep
state
pass in log quick proto tcp/udp from any to any port = 2049 keep frags
pass in log quick proto tcp from any to any port = 2121
pass in log quick proto tcp/udp from any to any port 49152 >< 65535
pass in log quick proto tcp from any to any port = 20
pass in log quick proto tcp from any to any port = 21
pass in log quick proto tcp from any to any port = 514
pass in log quick proto icmp from any to any icmp-type 8 keep state
pass in log quick proto tcp from any port = 514 to any keep state
pass in log quick proto tcp from any port = 1023 to any keep state

d) I would look at separate Ignite servers
for each customer. Smart customers will
accept the cost (a small price to pay for
a piece of mind).

Or, if each server has DVD drive (standard
these days), use latest Ignite bundles and
create DVDs...

Final advice: humans remember bad things
much longer than good ones. Avoid making
errors in design and implementation.

Sincerely,

VK2COT

[Bill Hassell]

One additional comment: If you have a dead machine and need to restore vg00 onto a new disk, most server models can only use the built-in LAN interface for bootup. So it won't matter how many NIC cards you have in each system, the Ignite recovery requires the built-in NIC to boot. Once booted (think: boot helper at the customer site), you have more options.

There's another issue - Ignite will be transferring unencrypted data during backup and restore. If this goes over the Internet, you are exposing company-private data. I would look at 11 Ignite net servers, one for each network.

The multi-DDS solution is the simplest but will require local operators to reliably collect, document and store the tapes.

[Team Leader UNIX DBA]

Hi all,

Yes, I agree that ending multiple clients on the same server network segment would not be acceptable.

It is clear that this is not going to be easy, Even using firewalls. Some clients only have one HP-UX server, so using a boot helper isn't going to work for them anyway...We will need some sort of boot media for those clients.

Correct me if I'm wrong, but I believe it is possible to create an Ignite boot media which boots a system, and then asks for network config, and from that point we can traverse routers/firewalls to get to the ignite-ux server, and the recovery archives? Is that right?

If so, then perhaps all we need to do is setup the firewall rules, and ensure each client server has a bootable media with the ignite-ux kernel on it etc. (eg DVD, or CD or even tape)

What do you think?

Each client is either located in our data centre, or is connected via VPN to our network.

Also, we already have a few "launchpad" servers from which we can connect to any of our client servers, eg via ssh. So it might be feasable to include the protocols Ignite-UX requires. That said, it appears Ignite-UX requires a lot of ports openned. Not so nice!!

Actually, it is starting to look a little too difficult to get a solid ignite-ux archive solution working for the customers.

As you say, I might have to approach it on a customer by customer basis.

Any other comments or ideas, please let me know. Thanks all.

- Andrew

[Bill Hassell]

> Correct me if I'm wrong, but I believe it is possible to create an Ignite boot media which boots a system, and then asks for network config, and from that point we can traverse routers/firewalls to get to the ignite-ux server, and the recovery archives? Is that right?

Correct. A boot helper is just an automated way to get a kernel into memory. Once the kernel is in place, routing can take place and any reachable network can be used to supply the rest of the image.

> If so, then perhaps all we need to do is setup the firewall rules, and ensure each client server has a bootable media with the ignite-ux kernel on it etc. (eg DVD, or CD or even tape)

Correct. And since these are secure connections, nothing else is needed. Regular backups can take place over the network and a restore can be done with a simple manual step (tape or CD) to startup.

[Armin Kunaschik]

> One additional comment: If you have a dead machine and need to restore
> vg00 onto a new disk, most server models can only use the built-in LAN
> interface for bootup.

This is not (fully) true for the Itanium machines. Even if the interfaces do not appear in the EFI menu you are able to boot from the network.
After the command "search all" on the EFI all LAN interfaces are recognized and a netboot works.

Beside that, I'd configure 2(to be able to restore the other server!!) ignite servers
at each costumer site (no central server).
They don't need to be standalone host, Ignite "just occupies space" if not active.
If the backups are not created at the same time, the NFS traffic will not be too high to impact normal server operation.

If you keep applications away from vg00, the ignite images will be no larger than 2-5GB and fit on local disks. Use "include" and "exclude"-options!
If your SLA demands it, the ignite images should be saved to tape (netbackup etc), even with a small retention time.

My 2 cents,
Armin