lbolt value error message.

[Aamir14]

Hi,

Yesterday I found some error message in syslog file.

prodata2#tail -10000000 /var/adm/syslog/syslog.log |grep CRITICAL

Aug  6 20:08:37 prodata2 EMS [3457]: ------ EMS Event Notification ------   Value: "CRITICAL (5)" for Resource: "/adapters/events/raid_adapter/0_6_0_0_0_0_4_0_0_0"     (Threshold:  >= " 3")    Execute the following command to obtain event details:   /opt/resmon/bin/resdata -R 226557954 -r /adapters/events/raid_adapter/0_6_0_0_0_0_4_0_0_0 -n 226557953 -a
prodata2#date
Tue Aug  7 13:02:04 IST 2012

prodata2#/opt/resmon/bin/resdata -R 226557954 -r /adapters/events/raid_adapter/0_6_0_0_0_0_4_0_0_0 -n 226557953 -a

CURRENT MONITOR DATA:

Event Time..........: Mon Aug  6 20:08:37 2012
Severity............: CRITICAL
Monitor.............: dm_raid_adapter
Event #.............: 43
System..............: prodata2

Summary:
     Adapter at hardware path 0/6/0/0/0/0/4/0/0/0 : CISS: Logical Drive
     transitioned state.



Description of Error:


 lbolt value: 181737283

    CISS: Logical Drive Status:
    CISS: Logical Drive 0 transitioned from OK to INTERIM RECOVERY
    Drive Type : RAID 1+0 - Disk Mirroring


Probable Cause / Recommended Action:

Informational message.

Additional Event Data:
     System IP Address...: 192.168.4.6
     Event Id............: 0x501fd6ed00000000
     Monitor Version.....: B.01.00
     Event Class.........: I/O
     Client Configuration File...........:
     /var/stm/config/tools/monitor/default_dm_raid_adapter.clcfg
     Client Configuration File Version...: A.01.00
          Qualification criteria met.
               Number of events..: 1
     Associated OS error log entry id(s):
          0x501fd6ed00000001
     Additional System Data:
          System Model Number.............: ia64 hp server rx6600
          OS Version......................: B.11.31
          EMS Version.....................: A.04.20.31.02
          STM Version.....................: D.04.00
     Latest information on this event:
          http://docs.hp.com/hpux/content/hardware/ems/dm_raid_adapter.htm#43

v-v-v-v-v-v-v-v-v-v-v-v-v    D  E  T  A  I  L  S    v-v-v-v-v-v-v-v-v-v-v-v-v



I/O Log Event Data:

     Driver Status Code..................: 0x0000002B
     Length of Logged Hardware Status....: 0 bytes.
     Offset to Logged Manager Information: 0 bytes.
     Length of Logged Manager Information: 184 bytes.

Manager-Specific Information:

Raw data from the SCSI RAID SA Controller CISS driver:
00000007 0AD51743 00000001 00000003 00000000 00000000 11165880 02000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000




prodata2#

Why this error generated.

How to resolve the problem

Thanks

Aamir

[rariasn]

Hi:

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&objectID=c00992076

rgs.

[Ken Grabowski]

Use the sasmgr command to view the status of disks and logical drives on your SAS controller.  Going from OK to interim recovery is a little strange. You normally recover from a failed state.  Are there any other messages in the log about your drive state?  You may have a disk that's starting to go bad, or a badly seated drive.

[Torsten.]

You most likely have a smartarray P400 controller installed. One disk of the RAID0+1 has failed.

Use ioscan to get the device file, e.g. /dev/ciss0

# ioscan -fnCciss

Then use sautil to get the status

# sautil /dev/ciss0

pull out the bad disk, insert the replacement and check again with sautil.

[Aamir14]

Hi,

This is my sautil output

prodata2#ioscan -fnC /dev/ciss0
prodata2#sautil /dev/ciss0

******************************************************************************
****                                                                      ****
****             S A U T I L   S u p p o r t   U t i l i t y              ****
****                                                                      ****
****             for the HP SmartArray RAID Controller Family             ****
****                                                                      ****
****                           version A.02.13                            ****
****                                                                      ****
****  (C) Copyright 2003-2007 Hewlett-Packard Development Company, L.P.   ****
******************************************************************************



---- DRIVER INFORMATION ------------------------------------------------------

  Driver State........................ READY

---- CONTROLLER INFORMATION --------------------------------------------------

  Controller Product Number........... P400
  Controller Product Name............. HP PCIe SmartArray P400
  Hardware Path....................... 0/6/0/0/0/0/4/0/0/0
  Device File......................... /dev/ciss0
  Hardware Revision................... 'E'
  Boot Block Revision................. 0.03
  Firmware Revision (running)......... 5.20
  Firmware Revision (in ROM).......... 5.20
  Firmware Revision (inactive)........ 5.22
  # of Logical Drives................. 1
  # of Physical Disks Configured...... 2
  # of Physical Disks Detected........ 1
  Logical Drive Rebuild Priority...... 0   (low)
  Array Capacity Expansion Priority... 0   (low)
  Auto-Fail Missing Disks at Boot..... disabled
  SCSI Transfer Detection Mode........ Auto Detect
  Controller Path Selection Method.... default


---- ARRAY ACCELERATOR (CACHE) INFORMATION -----------------------------------

  Array Accelerator Board Present?.... yes
  Cache Configuration Status.......... cache enabled
  Cache Ratio......................... 25% Read / 75% Write
  Total Cache Size (MB)............... 208
    Read Cache........................ 052
    Write Cache....................... 156
    Transfer Buffer................... 000
  Battery Pack Count.................. 1
  Battery Status (pack #1)............ ok


---- LOGICAL DRIVE SUMMARY ---------------------------------------------------

       #   RAID      Size        Status

       0   1+0      139979 MB    USING INTERIM RECOVERY MODE


---- SAS/SATA DEVICE SUMMARY -------------------------------------------------

  Location  Ct Enc Bay       WWID           Type        Capacity Status

  internal  2I   1   8  0x500000e01e9a0362  DISK       146.8 GB  OK
  N/A       2I   1   7  0x500000e01e9a1dd2  N/A        N/A       FAILED


---- SAS/SATA ENCLOSURE SUMMARY ----------------------------------------------

  Location  Ct   Enc  Expander_count    Bay_count     SEP_count

  internal  1I   1    0                 4             1
  internal  2I   1    0                 4             1


---- LOGICAL DRIVE 0 ---------------------------------------------------------

  Logical Drive Device File........... c0t0d0
  Fault Tolerance Mode................ RAID 1+0 (Disk Mirroring)
  Logical Drive Size.................. 139979 MB
  Logical Drive Status................ USING INTERIM RECOVERY MODE
  # of Participating Physical Disks... 2

  Participating Physical Disk(s)...... Ct:Enc:Bay:WWID
                                       2I:1:8:0x500000e01e9a0362
                                       2I:1:7:0x500000e01e9a1dd2 <-- NOT RESPONDING

  Participating Spare Disk(s)......... Ct:Enc:Bay:WWID
                                       none

  Stripe Size......................... 128 KB
  Logical Drive Cache Status.......... cache enabled
  Configuration Signature............. 0xA9CAF724
  Media Exchange Detected?............ no


---- SAS/SATA DEVICE 2I:1:8:0x500000e01e9a0362 [DISK] ------------------------

  Connector Location............................... internal
  Connector........................................ 2I
  Enclosure........................................ 1
  Bay.............................................. 8
  WWID............................................. 0x500000e01e9a0362
  Device Type...................................... DISK
  Disk Capacity.................................... 146.8 GB
  Device Status.................................... OK
  Device Vendor ID................................. HP
  Device Product ID................................ DG146BABCF
  Device Firmware Version.......................... HPD6
  Reserved Area (cfg/status info).................. 33554.4 KB (33.6 MB)
  Block Size (bytes/sector)........................ 512 bytes
  M&P Data Stamped?................................ yes
  Last Failure Reason.............................. none

  Physical Disk Flags:
    Disk present and operational................... yes
    Non-disk device detected....................... no

    S.M.A.R.T. supported........................... yes
    S.M.A.R.T. errors (in factory M&P data)........ no
    S.M.A.R.T. enabled............................. yes
    S.M.A.R.T. errors (in powerup M&P data)........ no
    Attached to external connector................. no
    Configured in a logical drive.................. yes
    Configured as a spare disk..................... no
    Disk write cache enabled at spin up............ no

    Supports redundant controller operation........ yes
    Disk write cache enabled in current page....... no
    Disk write cache disabled in default page...... yes





******************************************************************************
****                         End of SAUTIL Output                         ****
********************************************************

How to recover failed disk.

and how to find this disk is mirror or Primary disk.

Thanks 

Aamir

[Dennis Handly]

>How to recover failed disk?  and how to find this disk is mirror or Primary disk.

Participating Physical Disk(s)...... Ct:Enc:Bay:WWID
                                       2I:1:8:0x500000e01e9a0362
                                       2I:1:7:0x500000e01e9a1dd2 <-- NOT RESPONDING

You replace it.  I wouldn't think it matters which disk is "primary" for an array, especially when one failed.

[Aamir14]

Thanks for soonest reply

O.K I down the server

Than Insert new 146.8 GB disk.

one question ?

When insert a new disk on server its automatically mirroring,

or I will do some task or run some commands .

Thanks

Aamir

[Aamir14]

Hi,

I am only one engineer on my company to handle HP-UX servers

On my town and my company no one Knowing the HP-UX basic Knowledge

That's why I told you some basic questions .

Sorry some trouble.

Thanks

aamir

[Torsten.]

Do not shutdown the server - not needed!

Look at this:

 Location Ct Enc Bay WWID Type Capacity Status

internal 2I 1 8 0x500000e01e9a0362 DISK 146.8 GB OK
N/A 2I 1 7 0x500000e01e9a1dd2 N/A N/A FAILED

If you have a llok at your server front, you see a disk in slot 7 and in slot 8.

Disk in slot 7 is bad.

Pull the disk and insert the new disk.

Then check again with sautil - you should see the drive recovering.

[Aamir14]

I had shudown the server

during startup time its told press f1 to recover mode

after long process

 system started and its contained error message on syslog file.

tail -100000 /var/adm/syslog/syslog.log |grep error
Aug  8 00:06:24 prodata2 vmunix:       abort due to a write error on replacement disk 5:
prodata2#