HP-UX does not boot

[mahbub]

Dear all,

I am facing rp3440 server boot problem. Also unable to access BCH mode.

Processor is booting from the first available device.

To discontinue, press any key within 10 seconds.

I tried several times to discontinue by pressing any key but failed to access BCH mode.

finallay the below message showed
INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:ems4 "/etc/opt/resmon/lbin/p_client"

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:cim1 "/opt/wbem/lbin/cimserverd"

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:util "/usr/lbin/utild # Utilization Provider daemon"

I am attaching error message.

Any body help me to fix the problem.

Thanks and regards
Mahbub

[Michael Steele_2]

HI

Go back to the BCH, yes, that's what your attachement says, you were in the GSP, and 'bo alt'.

Did you mirror your boot disks?

Your problem appears to be the /etc/inittab file, did you modifity this file?

If not, what did you change in /etc?

You can also boot to single user. You can also boot to the old kernel.

[Dennis Handly]

It appears that /usr/bin/, etc. aren't mounting.

[mahbub]

Dear

Unable to access BCH mode.

To discontinue, press any key within 10 seconds.

I tried several times to discontinue by pressing any key but failed to access BCH mode.

How I will access the BCH mode?

Best regards
Mahbub

[Michael Steele_2]

HI

Can you control b and get to the GSP menu like in your attachement?

When you see the 'discontinue' message, I believe you're still seeing firmware generated messages, and this would indicate a system or gsp board. I believe they are two separate boards, with the GSP piggy backing onto the system board.

If you can't get back to the GSP with the control b command then I'm assuming a HW failure.

However, I have never seen or heard nor can I find any reference to this 'can not access the BCH error'.

[mahbub]

Dear,

Yes, I able to access BCH mode.
Please see the below information:

vlnboot
sh: vlnboot: not found
# lvlnboot -v
lvlnboot: Volume group not activated.
Cannot display volume group "/dev/vg00".
lvlnboot: Volume group not activated.
Cannot display volume group "/dev/vgtst".
lvlnboot: Volume group not activated.
Cannot display volume group "/dev/vgbkp".
# lvlnbfsck /dev/vg00/lvol3
vxfs fsck: Cannot open /dev/vg00/lvol3: No such device or address
file system check failure, aborting ...
# fsck /dev/vg00/lvol4
vxfs fsck: Cannot open /dev/vg00/lvol4: No such device or address
file system check failure, aborting ...
# bdf
sh: bdf: not found
# lvlnboot -R
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t0d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c2t1d0":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c6t0d1":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c8t0d1":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c6t0d2":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c6t0d3":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c6t0d4":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c8t0d2":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c8t0d3":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c8t0d4":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c6t0d5":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c6t0d6":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c8t0d5":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query physical volume "/dev/dsk/c8t0d6":
The specified path does not correspond to physical volume attached to
this volume group
lvlnboot: Warning: couldn't query all of the physical volumes.
vgcfgbackup: Warning: couldn't query physical volume "/dev/dsk/c6t0d5":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgbackup: Warning: couldn't query physical volume "/dev/dsk/c6t0d6":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgbackup: Warning: couldn't query physical volume "/dev/dsk/c8t0d5":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgbackup: Warning: couldn't query physical volume "/dev/dsk/c8t0d6":
The specified path does not correspond to physical volume attached to
this volume group
vgcfgbackup: Warning: couldn't query all of the physical volumes.
vgcfgbackup: Volume group not activated.


What will be next step? Please comment-----.


Best regards
Mahbub

[Michael Steele_2]

did you boot off the alt or the pri? single user or all the way up? Are any disks bad? Use 'dd=/dev/dsk/c t d of=/dev/null count=100'. What does i/oscan say? What does syslog.log say.

You got to know some of this? Start debugging.

[Vishu]

Hi,

Well, i think you are in maintenance mode or something where your VGs are not activated and you are trying to retrieve the information and getting those errors.

Try booting in single user mode and then try to mount your FS by activating all VGs and see if you have any bad disk in a particular VG which inhibits it from activating.

Regarding your first post, your error seems to come from /etc/inittab. Also, check for the same in that file.

[Michael Steele_2]

What is

'who -r'

[mahbub]

Dear all,

# diskinfo /dev/rdsk/c2t0d0
sh: diskinfo: not found
# strings /etc/lvmtab
sh: strings: not found
# vgdisplay -v vg00
--- Volume groups ---
VG Name /dev/vg00
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 11
Open LV 11
Max PV 16
Cur PV 2
Act PV 2
Max PE per PV 4384
VGDA 4
PE Size (Mbytes) 32
Total PE 8748
Alloc PE 7746
Free PE 1002
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg00/lvol1
LV Status available/syncd
LV Size (Mbytes) 320
Current LE 10
Allocated PE 20
Used PV 2

LV Name /dev/vg00/lvol2
LV Status available/syncd
LV Size (Mbytes) 4096
Current LE 128
Allocated PE 256
Used PV 2

LV Name /dev/vg00/lvol3
LV Status available/syncd
LV Size (Mbytes) 256
Current LE 8
Allocated PE 16
Used PV 2

LV Name /dev/vg00/lvol4
LV Status available/syncd
LV Size (Mbytes) 6016
Current LE 188
Allocated PE 376
Used PV 2

LV Name /dev/vg00/lvol5
LV Status available/syncd
LV Size (Mbytes) 10016
Current LE 313
Allocated PE 626
Used PV 2

LV Name /dev/vg00/lvol6
LV Status available/syncd
LV Size (Mbytes) 4032
Current LE 126
Allocated PE 252
Used PV 2

LV Name /dev/vg00/lvol7
LV Status available/syncd
LV Size (Mbytes) 2560
Current LE 80
Allocated PE 160
Used PV 2

LV Name /dev/vg00/lvol8
LV Status available/syncd
LV Size (Mbytes) 4608
Current LE 144
Allocated PE 288
Used PV 2

LV Name /dev/vg00/lvol9
LV Status available/syncd
LV Size (Mbytes) 30016
Current LE 938
Allocated PE 1876
Used PV 2

LV Name /dev/vg00/lvol10
LV Status available/syncd
LV Size (Mbytes) 50016
Current LE 1563
Allocated PE 3126
Used PV 2

LV Name /dev/vg00/lvol11
LV Status available/syncd
LV Size (Mbytes) 12000
Current LE 375
Allocated PE 750
Used PV 2


--- Physical volumes ---
PV Name /dev/dsk/c2t0d0
PV Status available
Total PE 4374
Free PE 501
Autoswitch On

PV Name /dev/dsk/c2t1d0
PV Status available
Total PE 4374
Free PE 501
Autoswitch On

# ioscan -fnC disk
Class I H/W Path Driver S/W State H/W Type Description
==========================================================================
disk 0 0/0/2/0.0.0.0 sdisk CLAIMED DEVICE TEAC DV-28E-N
/dev/dsk/c0t0d0 /dev/rdsk/c0t0d0
disk 1 0/1/1/0.0.0 sdisk CLAIMED DEVICE HP 146 GST3146707LC
/dev/dsk/c2t0d0 /dev/rdsk/c2t0d0
disk 2 0/1/1/0.1.0 sdisk CLAIMED DEVICE HP 146 GST3146707LC

# df
sh: df: not found
# bdf
sh: bdf: not found
# string /ec
sh: string: not found
# string /etc/lvmtab
sh: string: not found


HP-UX Start-up in progress
__________________________


VxVM general startup ................................................ OK
.................................................................... FAIL
*
Mount file systems .................................................. FAIL
*

Setting hostname .................................................... FAIL
*

Enable auxiliary swap space ......................................... FAIL
*

Configure Loopback interfaces (lo0) ................................. FAIL
*

Configuring all unconfigured software filesets ...................... FAIL
*
R
Configure HP igelan Gigabit Ethernet interfaces ..................... FAIL
*
Configure HP iether Gigabit Ethernet interfaces ..................... FAIL
*
Configure HP Gigabit Ethernet interfaces ............................ FAIL
*
Configure HP Ethernet interfaces .................................... FAIL
*

Configure LAN interfaces ............................................ FAIL
*
Start name server daemon ............................................ FAIL
*

Start NFS core subsystem ............................................ FAIL
*

Configuring PPP Interface ........................................... FAIL
*

Starting mail daemon ................................................ FAIL
*

Start SNMP Master Network Management daemon ......................... FAIL
*

Start SNMP HP-UNIX Network Management subAgent ...................... FAIL
*
Start SNMP MIB-2 Network Management subAgent ........................ FAIL
*
Start SNMP Trap Dest Network Management subAgent .................... FAIL
*
S
Start SNMP PCI FDDI Network Management subAgent ..................... FAIL
*

Initialize Instant Capacity ......................................... FAIL
*
.................................................................... FAIL
*

Starting HP Disk Array Manager daemons. ............................. FAIL
*
Starting disk array monitor daemons. ................................ FAIL
*

Start USB hub daemon ................................................ FAIL
*

Initialize Software Distributor agent daemon ........................ FAIL
*

Start EMS SNMP subagent ............................................. FAIL
*
Configure HP Fibre Channel Mass Storage interfaces .................. FAIL
*
Configure HP Internal RAID interfaces ............................... FAIL
*
Configuring Ultra320 SCSI Mass Storage interfaces ................... FAIL
*

preter "/usr/bin/sh" not found
FAIL *
Configure HP Fibre Channel Tachyon TL/TS Mass Storage interfaces .... FAIL
*
Configuring HP Fibre Channel FCD Mass Storage interfaces ............ FAIL
*
VVR - Starting Communication Daemon ................................. N/A
.................................................................... inter
preter "/bin/sh" not found
file link resolves to "/usr/bin/sh"
FAIL *
.................................................................... inter
preter "/bin/sh" not found
file link resolves to "/usr/bin/sh"
FAIL *
.................................................................... inter
preter "/bin/sh" not found
file link resolves to "/usr/bin/sh"
FAIL *
S
Start CDE login server .............................................. FAIL
*


* - An error has occurred !
* - Refer to the file /etc/rc.log for more information.



The system is ready.

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:cim1 "/opt/wbem/lbin/cimserverd"

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:cons "/usr/sbin/getty console console # system console"

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:util "/usr/lbin/utild # Utilization Provider daemon"

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:ems4 "/etc/opt/resmon/lbin/p_client"


Please inform me what will i do?

Best regards
Mahbub

[Michael Steele_2]

a) who -r
b) mount -a
c) bdf
d) grep -i err /etc/rc.log
e) dmesg

Get your file systems up! What's your run level?

[Dennis Handly]

>Michael: What is 'who -r'?

Unfortunately who(1) is broken and will always return 3 or greater.
But it is probably 3 since this rc(1m) error:
/sbin/rc3.d/S200tps.rc[15]: /usr/bin/hostname: not found

>b) mount -a
>Get your file systems up!

Right, these are most important to find out why this failed:
.................... FAIL *
Mount file systems ........ FAIL *
And why the first doesn't seem to have a name?

>d) grep -i err /etc/rc.log

This may help.

[mahbub]

Dear all,

Thanks for your mail.

INIT: Overriding default level with level 's'

INIT: SINGLE USER MODE

INIT: Running /sbin/sh

# who -r
sh: who: not found
# uname -a
HP-UX unknown B.11.11 U 9000/800 248600844 unlimited-user license
# mount -a
# bdf
sh: bdf: not found
# dmesg

Jan 3 07:30
Can't get kernel namelist
# grep -i err /etc/rc.log
sh: grep: not found



Thanks and regards
Mahbub

[Vishu]

Hi,

If you are in Single User mode, then first try mounting the /usr

# mount /usr
# bdf

First, set the hostname of your server with the command

# hostname
# mount -a

Then try changing the run level for the single user mode

# init 2
# init 3

[mahbub]

Dear all,


mount /usr--didn't mount
# bdf
sh: bdf: not found
# hostname
sh: hostname: not foun

I tried changing the run level for the single user mode but boot failed

What will be next step?

Best regards
Mahbub

[Vishu]

1)Can you tell me which run level you are in.
2) is /usr is a separate FS in your server.

okay, now from you vgdisplay of vg00, it is seen that you have two disks mirrored in your vg00. So, try booting your server from the alternate disk and see it the server is coming up.

if not again, then go to single user mode by

ISL> hpux -is -lq

[Dinesh Kumar B]

Mahbub

to get BCH mode type CO on the MP login

then u can able to try to boot from alt disk and previous kernel.

[mahbub]

Dear,

ISL> hpux /stand/vmunix.old

Boot
: disk(0/1/1/0.0.0.0.0.0.0;0)/stand/vmunix.old
disk(0/1/1/0.0.0.0.0.0.0;0)/stand/vmunix.old: cannot open, or not executable
Exec failed: No such file or directory.

Best Regards
Mahbub

[johnsonpk]

hi ,

On ISL prompt

execute
ISL> hpux ll
to list the contents of /stand and confirm that vmunix.old exist there

[Michael Steele_2]

Hi

a) Please explain the history of this box
b) Please explain the history of this crash - this is very important - don't leave anything out
c) After you perform 'johnsonpk's'
ISL> hpux ll...

please post the results by copy and paste.

Note that I'll be online most of the day if you'd like to work through this.

d) Do you have ignite backups?
e) Do you have data backups?
f) What application runs on this box?

[mahbub]

Dear all,

I didâ nt have any HW or SW changes on that server.
Regarding reboot, I was testing it to mount an NFS-shared drive from another windows server on it.
As it was a test server, then issued â rebootâ command.
But before rebooting I did â umount /frombkpâ , which unmounted my nfs mounted drive on this server (I have verified it by bdf)

2. from isl prompt:
ISL> hpux /stand/vmunix.prev--problem same
The system is ready.

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:cim1 "/opt/wbem/lbin/cimserverd"

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:util "/usr/lbin/utild # Utilization Provider daemon"

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:cons "/usr/sbin/getty console console # system console"

INIT: Command is respawning too rapidly.
Will try again in 5 minutes.
Check for possible errors.
id:ems4 "/etc/opt/resmon/lbin/p_client"


d) Do you have ignite backups?-No
e) Do you have data backups?no---
f) What application runs on this box?--Ora

Best Regards
Mahbub

[Michael Steele_2]

Hi

I'm going to give you some broad instructions to follow because you're performing things literally and not thinking for yourself.

a) You're object is to mount your file systems, especially /usr. Now you say you can do that. The only way that could happen is if the vg is not activated. But there have been no messages about this.

a1) mount /usr
a11) bdf -or- df -or- ls -la /usr
a2) if fails, 'vgchange -a y vg00'
a3) mount /usr
a31) see a11)

You got to get this up. Period. If you still can't, then you've deleted your /usr file system and again I have to ask you, what happened on this box to lose an entire file system?

b) vi /etc/inittab
b1) find the cim1 and p_client, and ems4, lines and comment them out.
-or-
b2) cp /etc/inittab /etc/inittab_old
b3) cp /usr/newconfig/etc/inittab /etc/inittab

You got to get this far. Period. If you do then

c) who -r

and update the thread.

d) Get some help over there from someone more knowledgeable. This is very basic stuff and you are failing at it.

[mahbub]

Dear All,

a1) mount /usr
a11) bdf -or- df -or- ls -la /usr
a2) if fails, 'vgchange -a y vg00'
a3) mount /usr
a31) see a11)
All above command output blank aqnd problem same. I am in singular use mode.
ISL>hpux -is -lq

what happened on this box to lose an entire file system?
Ans:
we didâ nt have any HW or SW changes on that server. Regarding reboot, I was testing it to mount an NFS-shared drive from another windows server on it. then isued â rebootâ command. But before rebooting I did â umount /frombkpâ , which unmounted my nfs mounted drive on this server (I have verified it by bdf).

b) vi /etc/inittab
b1) find the cim1 and p_client, and ems4, lines and comment them out.

I can't edit this file.do not work edit command or vi command.
-or-
b2) cp /etc/inittab /etc/inittab_old
b3) cp /usr/newconfig/etc/inittab /etc/inittab

Do not work cp command.

Thanks and regards
Mahbub

[Vishu]

Hey Mahbub,

Well, i have not seen you trying booting from your alternate disk path, which is a mirrored of your root disk. I already suggested you to do this.

I can see, you tried booting from same disk either in single user mode or using alternate kernel.

So, go ahead and try doing normal boot from your alternate disk path. If unsuccessful, do booting in single user mode using alternate disk only.

[mahbub]

Dear all,

I have tried to boot the system by using "bo pri' but the problem is same as bo pri.

Best regards
Mahbub