System hangs on boot at "Start NFS core system"

[BoMahony]

My system was recently rebooted and it is now having seriosu problems. The system is hanging on boot. when i break the boot sequence, it progresses as normal. However the NIC isnt working

/etc/rc.config.d/netconf:

HOSTNAME="moira"
OPERATING_SYSTEM=HP-UX
LOOPBACK_ADDRESS=127.0.0.1
INTERFACE_NAME[0]=lan0
IP_ADDRESS[0]=172.16.165.152
SUBNET_MASK[0]=255.255.248.0
BROADCAST_ADDRESS[0]=172.16.167.255
INTERFACE_STATE[0]=up
DHCP_ENABLE[0]=0
INTERFACE_MODULES[0]=""
ROUTE_SOURCE[0]=""
GATED=0
GATED_ARGS=""
RDPD=0
RARPD=0
DEFAULT_INTERFACE_MODULES=""
LANCONFIG_ARGS[0]=ether
ROUTE_DESTINATION[0]=default
ROUTE_GATEWAY[0]=172.16.164.1
ROUTE_COUNT[0]=1

bash-3.2# /sbin/init.d/net start    
sockfd error: : Host is down
ERROR:   Failed to set ndd tunable parameters
ERROR:   lan0 interface: ifconfig: socket: Host is down
ERROR:  route: socket: Host is down

I dont have a whole lot of experience troubleshooting HPUX so am at a loss where to start here.

what is see is the NIC isnt working. The lights on the back however are flashing. Attached is the rc.log file

[Matti_Kurkela]

Your rc.log includes several errors when setting up the loopback interface (lo0):

Configure Loopback interfaces (lo0)
Output from "/sbin/rc2.d/S008net.init start":
----------------------------
Boot time cleanup of /etc/ifconfig.muxids completed.
Message catalog can't be opened/accessed for language en_GB.iso88591.
Language C will used.
/usr/bin/autopush: Can't push requested modules on STREAM for entry 1
/usr/bin/autopush: Unexpected IOCTL(SAD_SAP/ ALL   ) error: 13
/usr/bin/autopush: Can't push requested modules on STREAM for entry 2
/usr/bin/autopush: Unexpected IOCTL(SAD_SAP/ ALL   ) error: 13
/usr/bin/autopush: Can't push requested modules on STREAM for entry 3
/usr/bin/autopush: Unexpected IOCTL(SAD_SAP/ ALL   ) error: 13
/usr/bin/autopush: Can't push requested modules on STREAM for entry 4
/usr/bin/autopush: Unexpected IOCTL(SAD_SAP/ ALL   ) error: 13
/usr/bin/autopush: Can't push requested modules on STREAM for entry 5
/usr/bin/autopush: Unexpected IOCTL(SAD_SAP/ ALL   ) error: 13
/usr/bin/autopush: Can't push requested modules on STREAM for entry 6
/usr/bin/autopush: Unexpected IOCTL(SAD_SAP/ ALL   ) error: 13
/usr/bin/autopush: Can't push requested modules on STREAM for entry 7
/usr/bin/autopush: Unexpected IOCTL(SAD_SAP/ ALL   ) error: 13
/usr/bin/autopush: Can't push requested modules on STREAM for entry 8
/usr/bin/autopush: Unexpected IOCTL(SAD_SAP/ ALL   ) error: 13
ERROR:   lo0 interface: ifconfig: socket: Host is down
sockfd error: : Host is down
ERROR:   Failed to set ndd tunable parameters
ERROR:   lo0 interface: ifconfig: IPv6 software has not been installed
"/sbin/rc2.d/S008net.init start" FAILED

 If lo0 is not properly configured, many things will fail. The errors above suggest problems with kernel module configuration.The root cause for lo0 configuration failure seems to be some sort of an issue with kernel modules.

• Is your /stand filesystem 100% full? It might have caused a previous kernel configuration operation to fail.
• What was the reason for the recent reboot? Was it a kernel configuration change, or did you add some patches?

Try rebooting with the previous kernel configuration; it should allow you to at least boot your system normally, unless your /stand filesystem is somehow damaged.

[BoMahony]

/stand                 (/dev/vg00/lvol1       ) :        9 % used blocks
                                                    337872 used blocks
                                                   3332144 free blocks

Reboot was due to users not knowing what they were doing with weblogic, and requesting a reboot (its a dev box).

>Try rebooting with the previous kernel configuration; it should allow you to at least boot your system normally, unless your /stand filesystem is somehow damaged.

How do i do this? dont have that much experience troubleshooting HPUX unfortunately.

[Matti_Kurkela]

The exact procedure depends on hardware model (mainly, is it PA-RISC or Itanium, and is it nPar-capable or not), HP-UX version and whether it is a physical or virtual machine.

Please show the output of these commands:

model
uname -a
setboot
parstatus
vparstatus
hpvmstatus

(Some of those commands might display errors, if your system is not virtualized or nPar-capable; that's not a problem.)

[BoMahony]

Apologies, I was on leave:

bash-3.2# model
ia64 hp server rx2620
bash-3.2# uname -a
HP-UX moira B.11.31 U ia64 3510053185 unlimited-user license
bash-3.2# setboot
Primary bootpath : 0/1/1/0.0x0.0x0 (/dev/rdisk/disk2)
HA Alternate bootpath :
Alternate bootpath : 0/0/2/0.0.0x0.0x0 (/dev/rdisk/disk3)

Autoboot is ON (enabled)
bash-3.2# parstatus
Note: No action specified. Default behavior is display all.
Internal Error: Received signal 11.
bash-3.2# vparstatus
bash: vparstatus: command not found
bash-3.2# hpvmstatus
bash: hpvmstatus: command not found

[Matti_Kurkela]

So... model rx2620, so an Itanium system, and no nPars, vPars or virtual machines to complicate things. Good.

See page 129 of this PDF document (the User Service Guide for rx2620):

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/action.process/public/psi/manualsDisplay/?sp4ts.oid=447331&javax.portlet.action=true&spf_p.tpst=psiContentDisplay&javax.portlet.begCacheTok=com.vignette.cachetoken&spf_p.prp_psiContentDisplay=wsrp-interactionState%3DdocId%253Demr_na-c01404139%257CdocLocale%253Den_US&javax.portlet.endCacheTok=com.vignette.cachetoken

(yes, the link is horribly long. I miss the old docs.hp.com.)

Follow the procedure titled "Booting HP-UX in Single User Mode (EFI Shell)", but in step 4, use "hpux -tm" instead of just "hpux". This will select the Tunable Maintenance Mode, which will ignore the current kernel configuration and use known-good default settings. If you use the suggested command line "boot -is vmunix" in step 5, the system will boot in single-user mode: as a result, only the root filesystem will be mounted, and even the root filesystem will be in read-only mode.

(Edit: Hmm, this document seems to suggest "boot -tm" in step 5 instead of "hpux -tm" in step 4, which would match the way to invoke the other recovery boot modes. But "man hpux.efi" on a 11.31 system seems to suggest "hpux -tm". Since I don't have a 11.31 system I could reboot at will at the moment, I'm afraid you'll have to try both versions.)

In single-user mode, once you reach the HP-UX root prompt on the system console, your first step should be the "mountall" command: it will remount the root filesystem in read-write mode and mount /usr and all the other important system filesystems, so you can start fixing things. If the system got this far with no errors, you might want to let the system boot up to the normal multi-user mode: the command for this would be "init 3". You should remember you're still running in Tunable Maintenance Mode, so the system performance may not be at the normal levels until you use the kernel configuration tools to regenerate a kernel configuration that is suited to your application requirements.

[BoMahony]

I followed your instructions up to "init 3", however it still fails here with the same problem:

Configure LAN interfaces ........................................................................................................................................................................ FAIL *

(after a few more hangs at :)

Start NFS core subsystem ........................................................................................................................................................................

I used :

Shell> fs0:

fs0:\> boot -tm
'boot' not found
Exit status code: Invalid Parameter

fs0:\> hpux -tm

(C) Copyright 1999-2006 Hewlett-Packard Development Company, L.P.
All rights reserved

HP-UX Boot Loader for IPF  --  Revision 2.035

Unrecognized command.

HPUX> boot -tm vmunix
> System Memory = 5108 MB
loading section 0

I then needed to do an fsck on the filesystems before i could mount them all.

Regardsless it still hung at the NFS component.

"so the system performance may not be at the normal levels until you use the kernel configuration tools to regenerate a kernel configuration that is suited to your application requirements."

Honestly i just need a basic kernel. No tuning really needs to be done, as the system is atest/dev box with ORACLE/WLS

I have never used these kernel configuration tools. Is there an easy way to reset the kernel to just a basic one, like right after an install?

Thanks for the help so far.

[BoMahony]

bash-3.2# kconfig
Configuration Title
backup Automatic Backup
last_install Created by last OS install
saved_4e678094
saved_4e678b01
saved_4e689f4a

I am now trying the backup, and last_install kernels

[BoMahony]

Seems both of those kernels fail also. Is there a HPUX utility that i can boot off CD and do a diags test, specificallly the network card?

Regards
B

[vijay alur alur]

Hello BoMahony,

Was this issue ever fixed for you?

i have similar issue, the server startup process hangs at the "Start NFS core system".

To Over come this i have set values to '0' for NFS_SERVER, NFS_CLIENT, NFS_CORE in /etc/rc.config.d.nfsconf. 

After this i am able to get the server boooted, but then it does not comes into network.

it gives error something like this for ipconfig

ifconfig: socket: Host is down

Appreciate you help!!

Regards,