tuxedo takes too long to shutdown and move to failover node

[Ron Madurski]

We have a tuxedo server with a failover node and a database server with a failover node in a cluster. During testing it was observed that a tuxedo only failover would take 15-20 minutes. Most of this time was spent in shutting down the tuxedo servers and closing their database connections.

An easy fix to this was to use a tmboot command to just kill the tux processes with no graceful cleanup. This speeds up the process significantly, but on some startups on the failover node not all of the tuxedo servers start backup because the database connections don't get cleaned up properly anymore.

Any suggestions on how to either:

1. Speed up the tuxedo shutdown.
2. Clean up the DB connections, quickly, before restarting.

Thanks,

Ron Madurski

[steven Burgess_2]

Hi Ron

Are you checking for any partitioned servers following the shutdown

echo psr | tmadmin -r | grep -i partion
echo psc | tmadmin -r | grep -i partion

What exactly are you doing to shutdown the servers ? tmshutdown -y ?

The only way to shut the services down are via a tmshutdown from the master server. A good follow up would be to clean the bullitin board via a bbc in tmadmin, following that check for any partitioning via the above.

Following a failover are you going to be rebooting the server you have failed over from ? if so you will need to clean up the partitioned entries with a

pcl

How many servers do you have running within your domain ? As everything resides in memory in can take an age to shut these things down so there really is no quick way to do it. If you get everything down cleanly then the alternate is going to boot a hell of a lot quicker and without any issues, otherwise it can all get very messy.


HTH

Steve

[steven Burgess_2]

whoops

I have spelt partition incorrectly in my command entries

echo psr | tmadmin -r | grep -i partition

etc

Steve

[Ron Madurski]

>>echo psr | tmadmin -r | grep -i partition
>>echo psc | tmadmin -r | grep -i partition

Next time we test this failover I (Or someone) will run these commands.


>>What exactly are you doing to shutdown the >>servers ? tmshutdown -y ?

We used to do a tmshutdown but that took way too long. Now it is set up to do a killtux. This shuts down quickly but sometimes it fails to come back up on the failover node due to database connections from the original node.

>>The only way to shut the services down are >>via a tmshutdown from the master server. A >>good follow up would be to clean the >>bullitin board via a bbc in tmadmin, >>following that check for any partitioning >>via the above.

>>Following a failover are you going to be >>rebooting the server you have failed over >>from ? if so you will need to clean up the >>partitioned entries with a

No plans to reboot the failed server as an SOP.

>>pcl

>>How many servers do you have running >>within your domain ? As everything resides >>in memory in can take an age to shut these >>things down so there really is no quick >>way to do it. If you get everything down >>cleanly then the alternate is going to >>boot a hell of a lot quicker and without >>any issues, otherwise it can all get very >>messy.


340 servers are supposed to start.

That is exactly what we see. Do a clean shutdown and the servers all start quickly and reliably. This takes about 15-20 minutes. Do a quick killtux and the servers start just as quickly but not as reliably :-(

[Ijajahmed]

We have a similar problem, how do we boot the processes quickly? We have over 500 processes to boot. Is there any way to make the tuxedo not disconnect but wait for connection in case of a db crash or fail over.

[Sunitha_Mod]

Hello @Ijajahmed, 

Thank you for posting! Since you have posted in an old topic and there is no response yet, I would recommend you to create a new topic using the create "New Discussion" button, so the experts can check and assist you further.