Databases and App server processes

[Michael Tully]

Hi,

I have an Informix database that resides on server A, and dependent application server processes (PeopleSloth) for it that live on server B. Now every week we do system reboots. (please don't ask why!) what I must do, is to make sure the application server processes that
are on server b are shutdown before the database on server A. Here's the hiccup. I have to have the database on server A back on line before the application server processes on server B start.

Suggestions....on the best way to achieve this...

Thanks
Michael

[Steven Sim Kok Leong]

Hi Michael,

On my site, we have Oracle database server (server A) and SAP application server (server B).

For daily offline database backups, the omniback pre-backup script on server A ssh's into server B, stops the application server before shutting down the database on server A. The omniback post-backup script on server A starts up the database on server A, then ssh's into server B and starts the application server.

I don't know if that is the best method but this is the method I have been using because it is secure (over ssh) and can be automated (based on public key authentication).

Hope this helps. Regards.

Steven Sim Kok Leong

[S.K. Chan]

Hi Michael,
The way I figure .. "single point of trigger" is the approach I would take. Within server B, i a single script ..
a) Shutdown/kill application processes
b) Remsh to server A and shutdown it's DB or reboot.
c) Monitors A's heartbeat, once it's up remsh to it again making sure DB started up fine.
d) Now reboot B.

just a thought ..

[Michael Tully]

Hi Steven and S.K. ,

Thanks for the input thus far. After posting my question I had another thought. We do our reboots at 5AM on a Sunday.

On server A at say 4:30AM we run a script
through cron that terminates the application
server processes.

At 5AM Server B (which has the databases)
gracefully reboots.

At 6AM Server A get rebooted.

I know there is no actual checking but I really
do want something simple. I don't want to go and install 'ssh' and I don't really to open up the systems to run 'remsh'. The part about the Application server being terminated first is the crucial part. (Mind you I wouldn't be fixing it either.... DBA problem....)

Any more thoughts??

Cheers
~Michael~

[Steven Sim Kok Leong]

Hi Michael,

After looking at S.K. Chan's reply, I realised that I have misread your query. :P

Two suggestions to add on to what S.K. has already mentioned:
1) use ssh (with public key authentication) instead of remsh for security reasons
2) if the script is part of a shutdown or startup script, it should not timeout too quickly while monitoring the heartbeat of the database server for availability.

Just my 2 cents. Hope this helps. Regards.

Steven Sim Kok Leong

[S.K. Chan]

Ok ..
1) 4.30am A's apps processes terminated
2) 5.00am B (DB srv) reboots
3) 6.00am A (apps srv) reboots
The time between 1 & 2 should be fine assuming 1 is always a guaranteed success, thus 2 need not check if A's processes are terminated before it reboots.
The tricky part is between 2 & 3, how to really make sure the the DB is up and running before you shutdown A (apps srv). I'm no DBA but I think you can run simple SQL queries from A to B just to verify the DB is up before A gets the reboot. If that's possible I would incorporate the SQL script to do the checking before step 3 is executed.

[S.K. Chan]

Hi (again) Michael , I missed the point you made .. "apps srv terminated first is the crucial part" ..
Which means at step 2, before DB srv is rebooted the check to make sure apps processes have been shutdown is more important. Is it suffice to assume if there is no DB connection then the apps processes have actually terminated ? If it is, again, a DBA ahould be able to tell you how to check that.

[Michael Tully]

Thanks again... I'll have a chat with the DBA and see what he wants to do. Ultimately it is his decision, as it's his responsibility. I'll leave this post open until tomorrow morning my time, now 4:45PM and nearly time to go home. (hit the toe! - aussie humour!)

Generous points will be awarded then, just in case someone has a brainstorm!

Cheers
~Michael~

[Tom Geudens]

Hi Michael,
Wow, some coordination problem. I would start by getting both the database and application start scripts out of the boots. Next I would use Control-M to accomplish the start/stop coordination. As an alternative (the chances that you have Control-M are rather small) a ITO/Openview automatic action would have to do the trick (which has the benefit over Control-M that you have the ITO tools to do all the checking/heartbeat stuff).

Without those softwares ... tricky. Does Informix have options (like Oracle) to execute external procedures (Operating System commands) ?

Hope this helps ... probably not but you asked for a brain storm :-)

Regards,
Tom

[Justo Exposito]

Hi Michael,

Do you have mail between this boxes?

You can define an aliases in the DB server that when receive a mail shutdown the database.

Then if you define that when the app process stop ok send a mail to the DB server and start the reboot process.

This is do it with somethin like that in the aliases file:
stopdb:"| scriptstopdb"

Other posibility is to ftp a control file to the DB server and when run the stop database check if the file is in the server.

Regards,

Justo.