lbolt logs in syslog right after cfg2html and lvlnboot execution ?

[Serge Poitras]

Hello HP aficionados,

We've noticed what appears to be a rather peculiar string of events from our syslog file, which are lbolt events right after executing the following automated (weekly) commands:

cfg2html and
lvlnboot -v /dev/vg00

After converting the HEX addresses, it's clear that the 2 devices affected here are the 2 internal 142gb drives.

At 1st glance it does not look like a genuine issue per say, but the occurance frequency does make me believe that something albe it minor, may need fixing ?

I have enclosed the actual patch listing on the server, even after cruising the ITRC for similar hits (none found), plus the latest event of this nature which by events in syslog, have started upon the last reboot (for maintenance, no new patches or kernel mods introduced) back in Feb 2009.


Sep 25 09:00:02 aph091 syslog: Start of Cfg2Html/HPUX Version 2.11
Sep 25 09:00:37 aph091 LVM[11988]: lvlnboot -v /dev/vg00
Sep 25 09:00:37 aph091 LVM[12032]: lvlnboot -v /dev/vg00
Sep 25 09:01:06 aph091 vmunix: SCSI: Data Overrun -- lbolt: 1948922877, dev: bc01f000
Sep 25 09:01:06 aph091 vmunix: lbp->state: 4060
Sep 25 09:01:06 aph091 vmunix: lbp->offset: ffffffff
Sep 25 09:01:06 aph091 vmunix: lbp->uPhysScript: f8040000
Sep 25 09:01:06 aph091 vmunix: From most recent interrupt:
Sep 25 09:01:06 aph091 vmunix: ISTAT: 0a, SIST0: 90, SIST1: 00, DSTAT: 80, DSPS: f80405c1
Sep 25 09:01:06 aph091 vmunix: lsp: 000000004b2b6f00
Sep 25 09:01:06 aph091 vmunix: bp->b_dev: bc01f000
Sep 25 09:01:06 aph091 vmunix: scb->io_id: 19ef253
Sep 25 09:01:06 aph091 vmunix: scb->cdb: 12 00 00 00 ff 00
Sep 25 09:01:06 aph091 vmunix: lbolt_at_timeout: 0, lbolt_at_start: 0
Sep 25 09:01:06 aph091 vmunix: lsp->state: 5
Sep 25 09:01:06 aph091 vmunix: lbp->owner: 0000000000000000
Sep 25 09:01:06 aph091 vmunix: scratch_lsp: 000000004b2b6f00
Sep 25 09:01:06 aph091 vmunix: Pre-DSP script dump [fffffffff8040348]:
Sep 25 09:01:06 aph091 vmunix: 60000008 00000000 60000040 00000000
Sep 25 09:01:06 aph091 vmunix: 0f000001 f80405c1 98080000 00000007
Sep 25 09:01:06 aph091 vmunix: Script dump [fffffffff8040368]:
Sep 25 09:01:06 aph091 vmunix: 0f000001 f80405c1 80840000 fffffea8
Sep 25 09:01:06 aph091 vmunix: 7c027f00 00000000 60000040 00000000
Sep 25 09:01:06 aph091 vmunix: lsp->puScript [0000000044001020]:
Sep 25 09:01:06 aph091 vmunix: 09000030 00c20080 09000040 00c98400
Sep 25 09:01:06 aph091 vmunix: 09000010 00ca0100 e25c0004 f80407ec
Sep 25 09:01:06 aph091 vmunix: SCSI: Data Overrun -- lbolt: 1948922877, dev: bc03f000
Sep 25 09:01:06 aph091 vmunix: lbp->state: 4060
Sep 25 09:01:06 aph091 vmunix: lbp->offset: ffffffff
Sep 25 09:01:06 aph091 vmunix: lbp->uPhysScript: f87bd000
Sep 25 09:01:06 aph091 vmunix: From most recent interrupt:
Sep 25 09:01:06 aph091 vmunix: ISTAT: 0a, SIST0: 90, SIST1: 00, DSTAT: 80, DSPS: f87bd5c1
Sep 25 09:01:06 aph091 vmunix: lsp: 00000000438a5500
Sep 25 09:01:06 aph091 vmunix: bp->b_dev: bc03f000
Sep 25 09:01:06 aph091 vmunix: scb->io_id: 34157c5
Sep 25 09:01:06 aph091 vmunix: scb->cdb: 12 00 00 00 ff 00
Sep 25 09:01:06 aph091 vmunix: lbolt_at_timeout: 0, lbolt_at_start: 0
Sep 25 09:01:06 aph091 vmunix: lsp->state: 5
Sep 25 09:01:06 aph091 vmunix: lbp->owner: 0000000000000000
Sep 25 09:01:06 aph091 vmunix: scratch_lsp: 00000000438a5500
Sep 25 09:01:06 aph091 vmunix: Pre-DSP script dump [fffffffff87bd348]:
Sep 25 09:01:06 aph091 vmunix: 60000008 00000000 60000040 00000000
Sep 25 09:01:06 aph091 vmunix: 0f000001 f87bd5c1 98080000 00000007
Sep 25 09:01:06 aph091 vmunix: Script dump [fffffffff87bd368]:
Sep 25 09:01:06 aph091 vmunix: 0f000001 f87bd5c1 80840000 fffffea8
Sep 25 09:01:06 aph091 vmunix: 7c027f00 00000000 60000040 00000000
Sep 25 09:01:06 aph091 vmunix: lsp->puScript [0000000044006000]:
Sep 25 09:01:06 aph091 vmunix: 09000030 00cc8580 09000040 00ce8400
Sep 25 09:01:06 aph091 vmunix: 09000010 00d08980 e25c0004 f87bd7f4

Thanks to all who will spend time and efforts in assisting me, points will as always be awarded upon relevancy.

Cheers,
Serge

[Michael Steele_2]

Hi

Lbolts are a head's up but not a failure. You can id the flaky disk easily with this command

cstm<<-EOF
runutil logtool
rs
EOF

Paste in the report from the Time Stamps (most important)

Thanks

[Serge Poitras]

Hello Michael and thank you for your suggestion. Here is the report output:

root @ aph091:HP-UX/11.11[/root]
# cstm << EOF
> runutil logtool
> rs
> EOF
Running Command File (/usr/sbin/stm/ui/config/.stmrc).

-- Information --
Support Tools Manager


Version A.52.00

Product Number B4708AA

(C) Copyright Hewlett Packard Co. 1995-2005
All Rights Reserved

Use of this program is subject to the licensing restrictions described
in "Help-->On Version". HP shall not be liable for any damages resulting
from misuse or unauthorized use of this program.

cstm>runutil logtool
-- Logtool Utility --
To View a Summary of Events in a Raw Log

1. Select a raw (unformatted) log file. (File Menu -> "Select Raw")
The current log file ends in ".cur", e.g. "log1.raw.cur".
You do not have to switch logs.

2. View the summary of the selected log file. (View Menu -> "Raw Summary")

To Format a Raw Log

1. Set the format filter for the types of entries you want to see.
(Filters Menu -> "Format"). To see all entries, skip this step.

2. Format the raw log file. (File Menu -> "Format Raw")

3. Display the formatted file. (View Menu -> "Formatted Log")

4. To further narrow the entries displayed, set a display filter.
(Filters Menu -> "Display" -> "Formatted")

For more information, use the on-line help (Help Menu -> "General help").

Logtool Utility>rs
Select Raw processing file /var/stm/logs/os/log277.raw.cur
Number of entries analyzed is 1.
Number of entries analyzed is 51.
Number of entries analyzed is 101.
Number of entries analyzed is 151.
Number of entries analyzed is 201.
Number of entries analyzed is 251.
Number of entries analyzed is 301.
Number of entries analyzed is 351.
Number of entries analyzed is 393.
The Select Raw operation completed.

The Display Raw Summary operation is currently in progress.

-- Converting a (2648) byte raw log file to text. --
Preparing the Logtool Utility: View Raw Summary File ...

.... aph091.wheaton.com : 10.11.244.91 ....

-- Logtool Utility: View Raw Summary --

Summary of: /var/stm/logs/os/log277.raw.cur

Date/time of first entry: Mon Mar 17 17:47:04 2008

Date/time of last entry: Mon Jun 1 06:05:04 2009



Number of LPMC entries: 0
Number of System Overtemp entries: 1
Number of LVM entries: 0
Number of Logger Event entries: 4

Number of I/O Error entries: 388


Device paths for which entries exist:

(5) 0/0/1/0.3.0
(1) 0/6/2/0.32.16.0.0.0.7
(1) 0/6/2/0.32.16.0.0.0.6
(1) 0/6/0/0.31.16.0.0.0.7
(1) 0/6/0/0.31.16.0.0.0.6
(18) 0/6/2/0
(51) 0/6/2/0.32.16.0.0.0.2
(46) 0/6/2/0.32.16.0.0.0.4
(62) 0/6/2/0.32.16.0.0.0.0
(55) 0/6/0/0
(48) 0/6/0/0.31.16.0.0.0.3
(49) 0/6/0/0.31.16.0.0.0.1
(50) 0/6/0/0.31.16.0.0.0.5

Products for which entries exist:

(5) Optical Storage Device
(73) Fibre Channel Interface
(33)
(277) SCSI Disk

Product Qualifiers for which entries exist:

(5) HPDVD-ROM
(73) HP6795A_Tachyon_XL2
(33)
(277) DGCCX3-80WDR5

Logger Events for which entries exist:

(73) td
(315) sdisk
-- Logtool Utility: View Raw Summary --
View - To View the file.
Print - To Print the file.
SaveAs - To Save the file.
Enter Done, Help, Print, SaveAs, or View: [Done] #.

[Michael Steele_2]

Hi

You have no emminent failure. Here's your report:

Date/time of first entry: Mon Mar 17 17:47:04 2008

Date/time of last entry: Mon Jun 1 06:05:04 2009

#############
Mar 17 to Jun 1 - Nothing since June. Nothing today. If there was a failure you'd have a 8 hour window and 500 errors logged.
#############

Number of LPMC entries: 0
Number of System Overtemp entries: 1
Number of LVM entries: 0
Number of Logger Event entries: 4

Number of I/O Error entries: 388


Device paths for which entries exist:

(5) 0/0/1/0.3.0
(1) 0/6/2/0.32.16.0.0.0.7
(1) 0/6/2/0.32.16.0.0.0.6
(1) 0/6/0/0.31.16.0.0.0.7
(1) 0/6/0/0.31.16.0.0.0.6
(18) 0/6/2/0
(51) 0/6/2/0.32.16.0.0.0.2
(46) 0/6/2/0.32.16.0.0.0.4
(62) 0/6/2/0.32.16.0.0.0.0
(55) 0/6/0/0
(48) 0/6/0/0.31.16.0.0.0.3
(49) 0/6/0/0.31.16.0.0.0.1
(50) 0/6/0/0.31.16.0.0.0.5

#########################
you are collecting errors on HW PATHs 0/6/2 and 0/6/0
#########################

Products for which entries exist:

(5) Optical Storage Device
(73) Fibre Channel Interface
(33)
(277) SCSI Disk

Product Qualifiers for which entries exist:

(5) HPDVD-ROM
(73) HP6795A_Tachyon_XL2
(33)
(277) DGCCX3-80WDR5

Logger Events for which entries exist:

(73) td
(315) sdisk
#########################
Optical drives are common sources of errors
#########################