Server take long time to reboot

[Vinod Kumar Yadav]

Hi Experts,

I am facing one issue from last few days.

The server, 2 Node HP-UX MCSG Cluster box with External EMC Storage Disk, is taking 12 hours for shutdown & reboot.

After it comes up, it works normal.

If anyone face this type of issue please help me out.

Thanks

[Patrick Wallek]

Where is the delay? At what point during shutdown and/or restart does it take a long time?

Is it during the stopping and/or starting of some of the processes via the rc scripts? If so, which ones?

What is the version of HP-UX you are running? What version of MC/SG? What kind of storage? What model server?

More information is almost always better.

[Mel Burslan]

One thing comes to mind: You have too many dead/unresponsive LUNs attached to your system. Try disconnecting the SAN prior to reboot and reconnect them after the reboot to see if it improves the timing. If so, your problem is definitely with your SAN. You can contact EMC to figure out bet way to deal with eliminating those dead links.

Another thing that comes to mind is networking issues but I have not had any experiences that network problems delaying my reboot in the order of hours let along 12 hours. Highly unlikely.

[Vinod Kumar Yadav]

Hi,

HP-UX O/S - 11.00
MC/Service Guard - A.11.14
Storage - EMC SYMMETRIX

If any information requried pls tell me.



Thanks

[Steven E. Protter]

Shalom,

/var/adm/syslog/

Take a look at the shutdownlog and see which step is taking time.

SEP

[Patrick Wallek]

>>Take a look at the shutdownlog and see which step is taking time.

That won't tell you anything other than who / what did the last shut down / reboot.

Questions that remain unanswered:

Where is the delay?
At what point during shutdown and/or restart does it take a long time?

Is it during the stopping and/or starting of some of the processes via the rc scripts? If so, which ones?

Logs to check for errors:

/etc/rc.log.old - for the previous startup and most recent shutdown
/etc/rc.log - for the startup processes

[Bill Hassell]

You need to add a timestamp for each step in the rc log. Edit the file: /sbin/rc.utils and change two lines:

Look for: echo "---
(there are 2 lines like that)

Change the line to:
echo "---------------------- $(date)" >> $LOGFILE

Noew you can see the time between each step. Almost positive the issue is with a bad SAN config or NFS that is broken.

[Steven E. Protter]

Another source to look at are the logs of the service guard packages.

This may be a problem with package fail over taking a long time.

SEP

[Roberto Arias]

please tryu this step for check:

1) disabled boot of cluster in boot server ( in /etc/rc.config.d/cmcluster put 0 in CMCLUSTER vrbl)

2) shutdown -ry and check /etc/rc.logwhen the network is started for look for erros in subsytems
(warning!! ¿can you stop the systems when you like it?)


If server boot more quickly than after, the problem is the serviguard cluster, check syslog.log and log of pkgs

Any pkg is not very fast in boot? check application of this pkg
regards

[TTr]

Do you have a busy (oracle) database running? If there are long batch jobs running an oracle database (with a "shutdown immediate") will wait until the jobs finish before shutting down.

[Vinod Kumar Yadav]

Hi,


After checking every thing this is very ecslated case.

The Backend team they recommend to install PATCH PHCO_26789


Thanks

[Patrick Wallek]

PHCO_26789 is a old patch (from April 2002) for HP-UX 11.0. Is this server an 11.0 server?

This patch has been superseded (replaced by) other patches twice.

I would actually install the most recent patch in this series: PHCO_32565 (Feb. 2006)
http://www11.itrc.hp.com/service/patch/patchDetail.do?patchid=PHCO_32565&sel={hpux:11.00,}&BC=main|search|

[Mel Burslan]

This patch was issued/released on 2002. ANd it is one of the most basic patches being sh-posix cumulative patch, which any patch bundle most probably included this one or something newer. Unless you are not patching your machines regularly, you should have a lter version of this patch. check the patch database for superseedy by information on this patch and make sure you really do not have anything.

And if you really do not have this patch or any superseeding patches installed, I'd strongly suggest you to review your patching policy.

[Vinod Kumar Yadav]

Hi,

Thanks I will update after patch installation.

PHCO_32565




Thanks
Vinod