System hangs upon shutdown - get some points here.

[dictum9]

Why does it hang on the NFS shutdown??
---------------------------------------------------------------------------------------------------------------------------------------



# shutdown -ry 0

SHUTDOWN PROGRAM
09/08/06 09:37:24 EDT

Broadcast Message from root (pts/1) Fri Sep 8 09:37:24...
PLEASE LOG OFF NOW ! ! !
System maintenance about to begin.
All processes will be terminated in 0 seconds.


Broadcast Message from root (pts/1) Fri Sep 8 09:37:24...
SYSTEM BEING BROUGHT DOWN NOW ! ! !


/sbin/auto_parms: DHCP access is disabled (see /etc/auto_parms.log)


/sbin/rc[16]: /sbin/rc1.d/K100hpnpd: not found.


System shutdown in progress
___________________________

Stop CDE login server ....................................... OK
Stopping HP-UX Apache-based Web Server ...................... OK
Stopping HP-UX Tomcat-based Servlet Engine. ................. OK
Stopping HP-UX Webmin-based Admin ........................... OK
Stopping the HPUX Webproxy subsystem ........................ OK
Stopping HP-UX XML Web Server Tools ......................... OK
Stop X print server(s) ...................................... N/A
Stop NFS server subsystem ................................... kk

[Alzhy · Accepted solution]

Do you have rc's that deal with possibly altering the network and something broke or was changed?

Have you tried monitoring rc.log on shutdown to see why it is stuck?

Do you shutdown from a non-console connection. Then on the console connection do:

tail -f rc.log


And see what's going on...


HTH>

[dictum9]

I ran the command 20 minutes ago and it's still hanging.

I got in at the console. I will try to stop the NFS daemons manually.

[Victor Fridyev]

Hi

It looks like the system can not unmount one of NFS. I had such a problem when Oracle or Informix processes started from NFS.

HTH

[dictum9]

Yeah, it is an Ignite server and is exporting a bunch of things to other machines (seen in /etc/exports)

Now what do I do at this point, if I just shutdown the NFS, won't it mean that in the future an NFS client will have to be rebooted?

[Pete Randall]

The first thing that should be shut down would be any applications: Oracle or other DB, for example. That way any files that the app has held open will be closed and the rest of the shutdown can proceed normally.


Pete

[Jaime Bolanos Rojas.]

Hi etc!

As you read through the posts, you will see that it can be several things, in my point of view if nothing was change to NFS or the scripts, I would think it's a patching issue.

Go to this link and see if you are missinng any patches for you OS.

http://www2.itrc.hp.com/service/patch/assessSystemsPage.do?BC=main%7C&admit=-1335382922+1157725147151+28353475

Regards,

Jaime.

[dictum9]

# tail -f rc.log
Stop X print server(s)
Output from "/sbin/rc2.d/K200tps.rc stop":
----------------------------
"/sbin/rc2.d/K200tps.rc stop" SKIPPED

Stop NFS server subsystem
Output from "/sbin/rc2.d/K900nfs.server stop":
----------------------------
killing nfsd
killing rpc.mountd

[Alzhy]

What does "showmount" tell you..? You mentioned this boxen is an Ignite server as well? Maybe it is just sooo busy serving your Ignite clients...

[dictum9]

# showmount
showmount: depotsvr: RPC: Program not registered

[Victor Fridyev]

Run
itical, run
#>bdf
and
fuser -cu {for all mount points}

Try to understand what process can not be killed in this situation.
If this is not something critical, run
#>reboot -q

HTH

[Pupil_1]

When you issued the shutdown command, there could have been some shares which are being accessed.

Whenever a NFS Server is shutdown or rebooted, the you will have to check for the nfs shares (exportfs) and then unmount all those shares in the client machines (list from /etc/rmtab in NFS Server) for a clean shutdown.

[dictum9]

fuser reveals that ignite is accessing /var/opt/ignite

I started ignite yesterday and it hung without even displaying the first screen. I thought I would give it time yet 12 hours later, it's still hanging. I thought I would reboot, and the reboot is hanging also as you can see. Same root cause of these?

[dictum9]

# ps -fe|grep ignite | grep -v grep
root 19775 1 0 19:17:10 ? 0:00 ignite
#
# kill -9 19775
# ps -fe|grep ignite | grep -v grep
root 19775 1 0 19:17:10 ? 0:00 ignite
#

[dictum9]

Now what....


# reboot -q
LVM: Recovered Path (device 0x1f092000) to PV 0 in VG 0.
LVM: vg[0]: pvnum=1 (dev_t=0x1f091000) is POWERFAILED
LVM: vg[0].lv[1] has 2 fault disks on : pvnum=0 (dev=0x1f092000) pvnum=1 (dev=0x1f091000)
LVM: vg[0].lv[2] has 2 fault disks on : pvnum=0 (dev=0x1f092000) pvnum=1 (dev=0x1f091000)
LVM: vg[0].lv[3] has 2 fault disks on : pvnum=0 (dev=0x1f092000) pvnum=1 (dev=0x1f091000)
LVM: vg[0].lv[4] has 2 fault disks on : pvnum=0 (dev=0x1f092000) pvnum=1 (dev=0x1f091000)LVM: vg[0].lv[8] cannot be close

[Pupil_1]

I think it's time to look for the reset button on the NFS server.

[Patrick Wallek]

Try doing a CTRL-SHIFT-| (that is the pipe symbol) and see if that will get you past the NFS issue.

When your system comes back up the NFS clients should be able to see their mountpoints again without any problem.

[Alzhy]

Etc,

It looks sure you have hardware issues... Disks that is...

Do the needful... reset (RS)...

[Pupil_1]

what's c9t2d0 and c9t1d0?

[dictum9]

Some weirdness.... looks like hardware issues?



To discontinue, press any key within 10 seconds.

10 seconds expired.
Proceeding...

Trying Primary Boot Path
------------------------

Booting... 2D10L
WARN A0BD
Boot Device not ready.
Retrying Boot Device...
Boot Device not ready.
Retrying Boot Device...
Boot Device not ready.
Retrying Boot Device...
Boot Device not ready.
Retrying Boot Device...
Boot Device not ready.
Retrying Boot Device...
Boot Device not ready.
Retrying Boot Device...
Boot Device not ready.
Retrying Boot Device...
Boot Device not ready.


Menu: Enter command or menu > search

Searching for potential boot device(s)
This may take several minutes.

To discontinue search, press any key (termination may not be immediate).

Path# Device Path (dec) Device Path (mnem) Device Type
----- ----------------- ------------------ -----------
P0 8/0/1/0.8 Fibre Channel interface
P1 8/4/1/0.8 Fibre Channel interface
P2 10/0.6 fwscsi.6 Random access media
P3 10/0.3 fwscsi.3 Random access media
P4 10/0.2 fwscsi.2 Random access media
P5 10/12/5.3 sescsi.3 Sequential access media
P6 10/12/5.2 sescsi.2 Random access media
P7 10/12/6.0 lan.0 LAN Module


Main Menu: Enter command or menu > boot p2
Interact with IPL (Y, N, or Cancel)?> n
C1D
Booting...
Boot IO Dependent Code (IODC) revision 152
2D10L
WARN C7F2
IPL error: bad LIF magic.
C1D2D10L
WARN C7F0
Error Reading IPL

CPU 0
WARNING: Memory has been initialized, but not tested as a result of
FASTBOOT being enabled. To test memory, use the "FASTBOOT"

[spex]

Looks like you have a bad disk...

PCS

[Sp4admin]

etc,

I would start checking the rc.log for any error that may expalin the problem. There may be a hardware problem.

sp,

[dictum9]

Boot Device not ready.
Retrying Boot Device... C1D2D10L
WARN 80F9
Cannot find ENTRY_TEST.

Failed to initialize.

ENTRY_INIT
Status = -7

FFFFFFF9 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000012 00000000 00000000 454E4841 00000000 00000000 454E4841
00000000 00000000 00000000 00000000 00000200 00000000 00000000 00000000
C1D2D10L
WARN 80F5
Failed to initialize.

C1D2D10L
WARN A008

CPU 0
WARNING: Memory has been initialized, but not tested as a result of
FASTBOOT being enabled. To test memory, use the "FASTBOOT"
command in the Configuration menu and reboot the system.

[Steven E. Protter]

Shalom etc,

Original problem: the stock oracle dbshut command issues a shutdown not a shutdown immediate. it needs to be changed if an oracle database has any hope of shutting down.

New Problem: You lost your boot disk. Have hardware replace the disk and prepare to boot off your Ignite Server and restore the vg00.

If you have an alternate boot disk, boot off of this at the ISL Prompt.

SEP

[dictum9]

Yeah, I lost my pri disk all right. I was able to boot off the alt. disk.