Operating System - HP-UX
1748169 Members
4471 Online
108758 Solutions
New Discussion

Stop Clusterware Services on graceful server shutdown

 
SOLVED
Go to solution
Raoof
Regular Advisor

Stop Clusterware Services on graceful server shutdown

Dear Experts,

 

We have  Oracle 11g  on HP-UX 11.31.

 

HP Unix is on itanium platform.

 

We configured UPS Power Protection Agent on the server, here i have a question that on automatic graceful shutdown by the UPS PPA will it also shutdown the database as well by stopping all the clusterware services which required to stop the database or not before shutting down the server.

 

Your advices will be highly appreciated.

 

Regards

Syed

10 REPLIES 10
Matti_Kurkela
Honored Contributor
Solution

Re: Stop Clusterware Services on graceful server shutdown

In general, it should work like this:

First, the PPA tells the OS to shutdown, typically using "shutdown -h now" or sending the necessary signals to the "init" process, so that it will begin the normal shutdown procedure.

 

Then, if the clusterware is properly installed, the standard startup/shutdown scripts in /sbin/rc?.d directories will cause the OS to tell the clusterware to shutdown using the appropriate commands, and wait for the clusterware shutdown to complete.

 

If the clusterware is properly configured, it should then tell the database to shutdown, using the appropriate commands, and wait for the database shutdown to complete. There might be a timeout: if the database cannot shutdown in a timely manner using the normal database shutdown commands, the clusterware might force the database to shutdown using other, more forceful commands.

 

When the database shutdown has been completed, the clusterware itself will complete its own shutdown procedures, and then the OS will be allowed to go on with the shutdown.

 

All this should happen exactly the same as when running "shutdown -h now" manually.

 

If you know that you need to shutdown the database and/or the clusterware services using some special commands without running "shutdown -h now", then someone has skipped some steps in the respective clusterware or database installation instructions. You should fix that before enabling PPA.

MK
Raoof
Regular Advisor

Re: Stop Clusterware Services on graceful server shutdown

Dear MK,

 

Thanks for your great and valuable reply with detail proccess of shutting down the server using PPA

 

Can you please just go through the below detail process of shutting down the server manually and check whether the clusterware is properly configured or not, if the below shutdown process is going in a proper way then is it PPA will shutdown the server in the same below manner.

 

mcmsdbl1#shutdown -h now

SHUTDOWN PROGRAM
10/28/11 04:32:39 WAT

Broadcast Message from root (pts/1) Fri Oct 28 04:32:39...
SYSTEM BEING BROUGHT DOWN NOW ! ! !


/sbin/auto_parms: DHCP access is disabled (see /etc/auto_parms.log)

    
     System shutdown in progress
     ___________________________

     Stop Oracle Cluster Ready Services .............................. OK
     Stop CDE login server ........................................... OK
     Stop LVM daemon ................................................. OK
     Stop kwdbd ...................................................... N/A
     Stopping HP-UX Apache-based Web Server .......................... OK
     Stopping HP-UX Tomcat-based Servlet Engine. ..................... N/A
     Stopping HP-UX Webmin-based Admin ............................... OK
     Stopping the HPUX Webproxy subsystem ............................ OK
     Stopping HP-UX XML Web Server Tools ............................. OK
     Stop X print server(s) .......................................... N/A
     Stop NFS server subsystem ....................................... OK
     Collecting AVIO and VIO hba and lun inventory.. ................. OK
     Shutting down CIFS Client ....................................... N/A
     Stopping Event Monitoring Service ............................... OK
     Stop interrupt balance daemon ................................... N/A
     Shutting down the Winbind Daemon ................................ OK
     Unload/Uninstall XF86 DLKM Helper Modules ....................... OK
     Stopping the gWLM Agent ......................................... OK
     Stopping Data Protector services ................................ OK
     Reset PRM and shut down PRM daemons ............................. OK
     Shutting down the System Management HomePage server ............. OK
     Stopping USB hub daemon ......................................... OK
     Stopping audio server daemon .................................... OK
     Stop auditing subsystem ......................................... OK
     Stop environment monitoring daemon .............................. OK
     Stop diagnostic subsystem ....................................... OK
     Stop oserrlogd .................................................. OK
     Stop clock daemon ............................................... OK
     Stop print spooler .............................................. OK
     Stop accounting ................................................. N/A
     Stopping time synchronization ................................... N/A
     Stop vt daemon .................................................. OK
     Terminating X Font Server ....................................... OK
     Stopping HP-UX Secure Shell ..................................... OK
     Stopping Instant Capacity ....................................... N/A
     Stop CIM cimserver subsystem .................................... OK
     Stop RPC daemon ................................................. OK
     Stop DCE daemons ................................................ OK
     Stopping OSPF MIB Network Management subAgent ................... N/A
     Stopping SNMP HP-UNIX Network Management subAgent ............... OK
     Stopping SNMP IPv6 Network Management subAgent .................. OK
     Stopping SNMP MIB-2 Network Management subAgent ................. OK
     Stopping Native Adapter Agent ................................... OK
     Stopping SNMP Trap Dest Network Management subAgent ............. OK
     Stopping SNMP Master Network Management daemon .................. OK
     Stopping outbound connection daemons for DDFA software .......... N/A
     Stopping syslog-ng daemons ...................................... N/A
     Stopping cfengine's cfservd daemon .............................. N/A
     Shutting down sendmail [Done] Shutting down sm-client [Done] .... OK
     Stopping IPv6 router advertisement daemon ....................... N/A
     Stopping remote system status daemon ............................ N/A
     Stopping RARP protocol daemon ................................... N/A
     Stop PPPoE configuration ........................................ N/A
     Stop PPP configuration .......................................... OK
     Stopping router discover protocol daemon ........................ N/A
     Stopping ramd routing daemon .................................... N/A
     Stopping dynamic routing daemon ................................. N/A
     Stopping Internet services daemon ............................... OK
     Stopping multicast routing daemon ............................... N/A
     Stop AUTOFS subsystem ........................................... OK
     Stop NFS client subsystem ....................................... OK
     Stop lock manager subsystem ..................................... OK
     Stop NIS client subsystem ....................................... OK
     Stop ldap client daemon ......................................... OK
     Stop NIS/LDAP server subsystem .................................. N/A
     Stop NIS server subsystem ....................................... OK
     Stop NFS core subsystem ......................................... OK
     Stopping name server daemon ..................................... N/A
     Unconfiguring DHCPv6 Interfaces ................................. OK
     Stopping DHCPv6 Server daemon ................................... N/A
     Stopping sfmdb PostgreSQL daemons ............................... OK
     Unconfigure LAN interfaces ...................................... OK
     Unconfigure LAN interfaces for IPv6 ............................. OK
     Unconfigure HP igssn Gigabit Ethernet interfaces ................ OK
     Unconfigure HP gelan Gigabit Ethernet interfaces ................ OK
     Unconfigure HP iether 100BT/Gigabit Ethernet interfaces ......... OK
     Unconfigure HP igelan Gigabit Ethernet interfaces ............... OK
     Stop network tracing and logging daemon ......................... OK
     Stop pty allocator daemon ....................................... OK
     Stop system message logging daemon .............................. OK
     Stop Software Distributor agent daemon .......................... OK
     Stop Essential Services monitoring daemon ....................... OK
     Stopping Event Management (EVM) ................................. OK
     Unconfigure Loopback interfaces (lo0) ........................... OK
     Stop Utmp Daemon ................................................ OK
     Stopping livedump subsystem ..................................... OK
     Stop syncer daemon .............................................. OK
     Killing user processes .......................................... OK
     Starting the fsdaemon for transition into single-user mode ...... OK
     Apply kernel configuration changes for next boot ................ N/A
     Unmount file systems ............................................ OK

Transition to run-level 0 is complete.

Wait for transition to run-level S (allow 20 seconds).

 

 

 

Thanks & Best Regards

Syed

Dennis Handly
Acclaimed Contributor

Re: Stop Clusterware Services on graceful server shutdown

>if the below shutdown process is going in a proper way

 

It seems like it since:

Stop Oracle Cluster Ready Services ......... OK

Raoof
Regular Advisor

Re: Stop Clusterware Services on graceful server shutdown

Dear Dennis,

 

Thanks alot for your reply. I can now proceed further for configuring PPA on the server.

 

Thanks to MK also for replying with great explanation.

 

Regards

Raoof
Regular Advisor

Re: Stop Clusterware Services on graceful server shutdown

Dear Matti,

 

Hope you are doing good, sorry for troubling you again. Actually i just want to update you that the PPA which we configured on the server worked very well for the process of shutting down the server gracefully.

But the server did not comes up automatically once the power was on, we startup the server manually.

 

So here my concern is why the ppa did not startup the server after the power was on. Do i need to go with any further steps to get success in that process of startup.

 

Also i will be really grateful if you could please let me know whether the server(unix) will write down the shutdown process in any log file with which i can come to know the details of the server which got down successfully in the graceful manner.

 

like below shutdown process can be seen at the time of shutting down server manually.

mcmsdbl1#shutdown -h now

SHUTDOWN PROGRAM
10/28/11 04:32:39 WAT

Broadcast Message from root (pts/1) Fri Oct 28 04:32:39...
SYSTEM BEING BROUGHT DOWN NOW ! ! !


/sbin/auto_parms: DHCP access is disabled (see /etc/auto_parms.log)

   
     System shutdown in progress
     ___________________________

     Stop Oracle Cluster Ready Services .............................. OK
     Stop CDE login server ........................................... OK
     Stop LVM daemon ................................................. OK
     Stop kwdbd ...................................................... N/A
     Stopping HP-UX Apache-based Web Server .......................... OK
     Stopping HP-UX Tomcat-based Servlet Engine. ..................... N/A
     Stopping HP-UX Webmin-based Admin ............................... OK

 

Your valuable advice and suggestions will be highly appriciated and will be really grateful for you kind assistance aslo.

 

Regards

 

Matti_Kurkela
Honored Contributor

Re: Stop Clusterware Services on graceful server shutdown

The PPA cannot restart the server when power returns, because it is just a piece of software: when a system is shut down, it runs no software at all. The restart when power returns requires a combination of settings, both in the UPS and in the server firmware.

 

Another issue is that restarting the server immediately after power comes back may not be a very good idea: if the power fails again soon afterwards, the server would again fall back to the power from the UPS. If this happens several times in a row, it would cause unnecessary wear to both the UPS and the server itself. (Remember that if power is lost while the server is still starting up, the PPA might not have started yet, so the server cannot begin shutdown until it has completed starting... so it must remain on the UPS power until the PPA can shut it down again. With a big server, this may take a lot of time, so you'll really want to avoid false starts.)

 

Many UPSes can be configured to wait a certain number of seconds or minutes after power returns before they start delivering power to the devices they're protecting. This provides two advantages: 1.) it does not cause the UPS battery to be drained further by another controlled shutdown if the power fails again within the time delay, and 2.) when the power comes back, the delay protects the server from the power surges and spikes caused by a lot of electric motors and other devices starting up at the same time.

 

So, you'll need to decide a suitable power-on delay for your environment, and configure that to your UPS.

Then you'll need to make sure your servers are configured to start automatically as soon as UPS starts giving them power again.

 

The shutdown process is logged at /etc/rc.log (which will be renamed to /etc/rc.log.old as soon as the next startup process gets as far as mounting the root filesystem in read/write mode). When something is displayed as "FAIL" in the shutdown/startup display, this log file will contain the actual error message.

 

The startup process is also logged in /etc/rc.log, but since a lot of things need to happen before the system can even access the root filesystem, you may need to also look at the "console log"/"console history" function of the Management Processor. This will capture the early boot messages displayed on the system console.

MK
Raoof
Regular Advisor

Re: Stop Clusterware Services on graceful server shutdown

Dear Matti,

 

I would like to thank you so much for taking the time to answer me in detail. Your input means a lot. Thank you! Highly appreciated!. Thanks for your detail explanation and Sorry for comming back late and also not gone with much more details in my last reply regarding the time intervals which we configured in the ups.


In the ups we had configured the time to wait for shutting down the server after the power goes off is about 20 minutes and even it should wait upto 20 minutes to restart the server once the porwer returns. So the process of shutting down the server was completed in a precise manner. But the server not come up after the power returns on which you said "The PPA cannot restart the server when power returns, because it is just a piece of software: when a system is shut down, it runs no software at all." on this i totally agreed with your statement.

Which also gives the clear logical view that definetly it will requires a combination of settings, both in the UPS and in the server also. With this awareness only i request for the configuration's which needs to be apply at the serverl level.

 

About your statement on "Another issue" so dear i suare i really didn't mean to say that why the ppa did not start the server immediatly after power comes back when i said "why the ppa did not startup the server after the power was on".


Anyways, now i really need your assistance to configure the server in such a way that it should start automatically as soon as UPS starts giving them power again. As for now when the power returns, ups comes up and transfering the power to the server only, after which server should get restart automatically after proper configuration as you said.


So, please advice me for further process.


Thanks & Best Regards

Matti_Kurkela
Honored Contributor

Re: Stop Clusterware Services on graceful server shutdown

OK. I would need to know your server model, then.

Please run the command "model" on your HP-UX system and show the output.

MK
Raoof
Regular Advisor

Re: Stop Clusterware Services on graceful server shutdown

Thanks for your assistance. Please find below the output of the requested command.

 

model

 

ia64 hp server rx3600