Operating System - HP-UX
1748102 Members
5444 Online
108758 Solutions
New Discussion

Re: lbolt value error message.

 
Aamir14
Regular Advisor

lbolt value error message.

Hi,

 

Yesterday I found some error message in syslog file.

 

prodata2#tail -10000000 /var/adm/syslog/syslog.log |grep CRITICAL


Aug  6 20:08:37 prodata2 EMS [3457]: ------ EMS Event Notification ------   Value: "CRITICAL (5)" for Resource: "/adapters/events/raid_adapter/0_6_0_0_0_0_4_0_0_0"     (Threshold:  >= " 3")    Execute the following command to obtain event details:   /opt/resmon/bin/resdata -R 226557954 -r /adapters/events/raid_adapter/0_6_0_0_0_0_4_0_0_0 -n 226557953 -a
prodata2#date
Tue Aug  7 13:02:04 IST 2012

 

 

prodata2#/opt/resmon/bin/resdata -R 226557954 -r /adapters/events/raid_adapter/0_6_0_0_0_0_4_0_0_0 -n 226557953 -a

CURRENT MONITOR DATA:

Event Time..........: Mon Aug  6 20:08:37 2012
Severity............: CRITICAL
Monitor.............: dm_raid_adapter
Event #.............: 43
System..............: prodata2

Summary:
     Adapter at hardware path 0/6/0/0/0/0/4/0/0/0 : CISS: Logical Drive
     transitioned state.



Description of Error:


 lbolt value: 181737283

    CISS: Logical Drive Status:
    CISS: Logical Drive 0 transitioned from OK to INTERIM RECOVERY
    Drive Type : RAID 1+0 - Disk Mirroring


Probable Cause / Recommended Action:

Informational message.

Additional Event Data:
     System IP Address...: 192.168.4.6
     Event Id............: 0x501fd6ed00000000
     Monitor Version.....: B.01.00
     Event Class.........: I/O
     Client Configuration File...........:
     /var/stm/config/tools/monitor/default_dm_raid_adapter.clcfg
     Client Configuration File Version...: A.01.00
          Qualification criteria met.
               Number of events..: 1
     Associated OS error log entry id(s):
          0x501fd6ed00000001
     Additional System Data:
          System Model Number.............: ia64 hp server rx6600
          OS Version......................: B.11.31
          EMS Version.....................: A.04.20.31.02
          STM Version.....................: D.04.00
     Latest information on this event:
          http://docs.hp.com/hpux/content/hardware/ems/dm_raid_adapter.htm#43

v-v-v-v-v-v-v-v-v-v-v-v-v    D  E  T  A  I  L  S    v-v-v-v-v-v-v-v-v-v-v-v-v



I/O Log Event Data:

     Driver Status Code..................: 0x0000002B
     Length of Logged Hardware Status....: 0 bytes.
     Offset to Logged Manager Information: 0 bytes.
     Length of Logged Manager Information: 184 bytes.

Manager-Specific Information:

Raw data from the SCSI RAID SA Controller CISS driver:
00000007 0AD51743 00000001 00000003 00000000 00000000 11165880 02000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000




prodata2#

Why this error generated.

 

How to resolve the problem

 

Thanks

Aamir

 

 

 

31 REPLIES 31
rariasn
Honored Contributor

Re: lbolt value error message.

Ken Grabowski
Respected Contributor

Re: lbolt value error message.

Use the sasmgr command to view the status of disks and logical drives on your SAS controller.  Going from OK to interim recovery is a little strange. You normally recover from a failed state.  Are there any other messages in the log about your drive state?  You may have a disk that's starting to go bad, or a badly seated drive.

Torsten.
Acclaimed Contributor

Re: lbolt value error message.

You most likely have a smartarray P400 controller installed. One disk of the RAID0+1 has failed.

 

Use ioscan to get the device file, e.g. /dev/ciss0

 

# ioscan -fnCciss

 

Then use sautil to get the status

 

# sautil /dev/ciss0

 

pull out the bad disk, insert the replacement and check again with sautil.


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
Aamir14
Regular Advisor

Re: lbolt value error message.

Hi,

 

This is my sautil output

prodata2#ioscan -fnC /dev/ciss0
prodata2#sautil /dev/ciss0

******************************************************************************
****                                                                      ****
****             S A U T I L   S u p p o r t   U t i l i t y              ****
****                                                                      ****
****             for the HP SmartArray RAID Controller Family             ****
****                                                                      ****
****                           version A.02.13                            ****
****                                                                      ****
****  (C) Copyright 2003-2007 Hewlett-Packard Development Company, L.P.   ****
******************************************************************************



---- DRIVER INFORMATION ------------------------------------------------------

  Driver State........................ READY

---- CONTROLLER INFORMATION --------------------------------------------------

  Controller Product Number........... P400
  Controller Product Name............. HP PCIe SmartArray P400
  Hardware Path....................... 0/6/0/0/0/0/4/0/0/0
  Device File......................... /dev/ciss0
  Hardware Revision................... 'E'
  Boot Block Revision................. 0.03
  Firmware Revision (running)......... 5.20
  Firmware Revision (in ROM).......... 5.20
  Firmware Revision (inactive)........ 5.22
  # of Logical Drives................. 1
  # of Physical Disks Configured...... 2
  # of Physical Disks Detected........ 1
  Logical Drive Rebuild Priority...... 0   (low)
  Array Capacity Expansion Priority... 0   (low)
  Auto-Fail Missing Disks at Boot..... disabled
  SCSI Transfer Detection Mode........ Auto Detect
  Controller Path Selection Method.... default


---- ARRAY ACCELERATOR (CACHE) INFORMATION -----------------------------------

  Array Accelerator Board Present?.... yes
  Cache Configuration Status.......... cache enabled
  Cache Ratio......................... 25% Read / 75% Write
  Total Cache Size (MB)............... 208
    Read Cache........................ 052
    Write Cache....................... 156
    Transfer Buffer................... 000
  Battery Pack Count.................. 1
  Battery Status (pack #1)............ ok


---- LOGICAL DRIVE SUMMARY ---------------------------------------------------

       #   RAID      Size        Status

       0   1+0      139979 MB    USING INTERIM RECOVERY MODE


---- SAS/SATA DEVICE SUMMARY -------------------------------------------------

  Location  Ct Enc Bay       WWID           Type        Capacity Status

  internal  2I   1   8  0x500000e01e9a0362  DISK       146.8 GB  OK
  N/A       2I   1   7  0x500000e01e9a1dd2  N/A        N/A       FAILED


---- SAS/SATA ENCLOSURE SUMMARY ----------------------------------------------

  Location  Ct   Enc  Expander_count    Bay_count     SEP_count

  internal  1I   1    0                 4             1
  internal  2I   1    0                 4             1


---- LOGICAL DRIVE 0 ---------------------------------------------------------

  Logical Drive Device File........... c0t0d0
  Fault Tolerance Mode................ RAID 1+0 (Disk Mirroring)
  Logical Drive Size.................. 139979 MB
  Logical Drive Status................ USING INTERIM RECOVERY MODE
  # of Participating Physical Disks... 2

  Participating Physical Disk(s)...... Ct:Enc:Bay:WWID
                                       2I:1:8:0x500000e01e9a0362
                                       2I:1:7:0x500000e01e9a1dd2 <-- NOT RESPONDING

  Participating Spare Disk(s)......... Ct:Enc:Bay:WWID
                                       none

  Stripe Size......................... 128 KB
  Logical Drive Cache Status.......... cache enabled
  Configuration Signature............. 0xA9CAF724
  Media Exchange Detected?............ no


---- SAS/SATA DEVICE 2I:1:8:0x500000e01e9a0362 [DISK] ------------------------

  Connector Location............................... internal
  Connector........................................ 2I
  Enclosure........................................ 1
  Bay.............................................. 8
  WWID............................................. 0x500000e01e9a0362
  Device Type...................................... DISK
  Disk Capacity.................................... 146.8 GB
  Device Status.................................... OK
  Device Vendor ID................................. HP
  Device Product ID................................ DG146BABCF
  Device Firmware Version.......................... HPD6
  Reserved Area (cfg/status info).................. 33554.4 KB (33.6 MB)
  Block Size (bytes/sector)........................ 512 bytes
  M&P Data Stamped?................................ yes
  Last Failure Reason.............................. none

  Physical Disk Flags:
    Disk present and operational................... yes
    Non-disk device detected....................... no

    S.M.A.R.T. supported........................... yes
    S.M.A.R.T. errors (in factory M&P data)........ no
    S.M.A.R.T. enabled............................. yes
    S.M.A.R.T. errors (in powerup M&P data)........ no
    Attached to external connector................. no
    Configured in a logical drive.................. yes
    Configured as a spare disk..................... no
    Disk write cache enabled at spin up............ no

    Supports redundant controller operation........ yes
    Disk write cache enabled in current page....... no
    Disk write cache disabled in default page...... yes





******************************************************************************
****                         End of SAUTIL Output                         ****
********************************************************

 

 

How to recover failed disk.

and how to find this disk is mirror or Primary disk.

 

Thanks 

Aamir

 

 

 

Dennis Handly
Acclaimed Contributor

Re: RAID drive failed error message

>How to recover failed disk?  and how to find this disk is mirror or Primary disk.

Participating Physical Disk(s)...... Ct:Enc:Bay:WWID
                                       2I:1:8:0x500000e01e9a0362
                                       2I:1:7:0x500000e01e9a1dd2 <-- NOT RESPONDING

 

You replace it.  I wouldn't think it matters which disk is "primary" for an array, especially when one failed.

 

 

 

Aamir14
Regular Advisor

Re: RAID drive failed error message

Thanks for soonest reply

 

O.K I down the server

Than Insert new 146.8 GB disk.

 

one question ?

When insert a new disk on server its automatically mirroring,

or I will do some task or run some commands .

 

Thanks

Aamir

 

 

 

 

Aamir14
Regular Advisor

Re: RAID drive failed error message

Hi,

 

I am only one engineer on my company to handle HP-UX servers

 

On my town and my company no one Knowing the HP-UX basic Knowledge

 

That's why I told you some basic questions .

Sorry some trouble.

 

Thanks

aamir

Torsten.
Acclaimed Contributor

Re: RAID drive failed error message

Do not shutdown the server - not needed!

 

 

Look at this:

 

 Location Ct Enc Bay WWID Type Capacity Status

internal 2I 1 8 0x500000e01e9a0362 DISK 146.8 GB OK
N/A 2I 1 7 0x500000e01e9a1dd2 N/A N/A FAILED

If you have a llok at your server front, you see a disk in slot 7 and in slot 8.

 

Disk in slot 7 is bad.

 

Pull the disk and insert the new disk.

 

Then check again with sautil - you should see the drive recovering.

 

 


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
Aamir14
Regular Advisor

Re: RAID drive failed error message

I had shudown the server

 

during startup time its told press f1 to recover mode

after long process

 

 system started and its contained error message on syslog file.

 

tail -100000 /var/adm/syslog/syslog.log |grep error
Aug  8 00:06:24 prodata2 vmunix:       abort due to a write error on replacement disk 5:
prodata2#