Instant TopTools event on LH3

[Tom Hatfield_1]

I am running the latest T.T.agents (5.54)on a Netserver LH3. Front Panel LED blinking Red as well as a drive LED. Instant T.T. indicates the following critical message.

21142 SCSI ScsiSMART: Device failed: host adapter 1, Target ID 0 and LUN 0
Code Message:One of the devices which was installed in the server can no longer be detected by the SCSIFLT

Then the following message.

21144 SCSI ScsiSMART: Device recovered: host adapter 1, Target ID 0 and LUN 0.
Code Message: This device is now accessible again and/or this device is replaced by an identical device. This message is for information purposes only. No action is necessary.

I have ran the diagnotics listed on the on the 21142 code including drives and SCSI bus and get no errors.

This process repeats itself about every fifteen minutes. Device failed....then...Devise recovered.

What could be wrong?

[Lex van Haaster]

Tom, have you ever found a resolution to this? We're experiencing almost exactly the same symptoms on one of our LH3 Netservers, only difference here is the events reappear every 30 minutes, not 15. Other than these events being logged, there seem to be no operational problems with it though.

[amhakassa]

Hi Tom and Lex Van,

If you have netraid assistant check the drives status with in Netraid assistant. You can check the hard drives physical status and see if there is any media or non media errors or predictive failure

If you don't have Netraid assistant you can download and install it from this link
http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?locale=en_US&pnameOID=15389&taskId=135&prodTypeId=15351&prodSeriesId=50448&submit.y=0&submit.x=0〈=en&cc=us

1. Select you Operating System
2. Download the Utility found under "Software Storage Controller"
3. Unzip it to a folder and run the Setup file with in that folder (Pick manage this server only)

Note:
You will be asked to restart the server. If you do not want to do so you can say NO and the Utility will still work.

You can run it by going to Start/Programs/Netraid Assistant/Full Access

Regards
Amha Kassa

[Leonard Davison]

With the LH3's there was an issue with the SCSI backplane and BIOS. It would report failed drives and/or fail drives that there was nothing wrong with. The only way I have cleared these errors was to:
A) Make sure ALL bios are up to date including SCSI disk.
B.) Replacement of the SCSI Backplane. This may not be a cheap option if the unit is out of warranty.

HP engineers could never give me an adequate answer for why the LH3 and LC2000 would consistantly do this but usually step A. would resolve 95% of these issues.

[Terri Harris]

There was & is a specific issue with the TopTools SCSI Smart Detail Agent. This agent needs to be uninstalled, otherwise it will erroneously report errors.
'
1. Uninstall existing NetServer agents.
2. Do a Custom install of the latest Netserver agents. This provides the option of unchecking the SCSI Detail Smart Agent.

This allows you to use TopTools without getting the repeating "SCSI ScsSMART device recovered" errors.

[Lex van Haaster]

Thanks all, for now my problem is solved..
Performing the several different checks didn't show up anything, and we have upgraded all the firmwares and bios to the latest version without succes.
Removing the scsi smart & detail agent was the solution however. Since nothing else seems to be wrong, and this particular system will be replaced in a few months anyways, it's a good enough workaround for us.

Regards,
Lex van Haaster

[Pat Lefemine_1]

I am having the same problem right now with my LC3. It just started yesterday after I rebooted.

My question is this, for the solution you guys found. above, did your drive lights flash red before you removed the smart detail agent?

Also, where do I find these netserver agents and how do I remove them?

Thanks in advance.

Pat

[Lex van Haaster]

Yes actually the drive-led's were flashing red in my case. Clearing the toptools event-log would stop it for a while, but the drives would start flashing red again as soon as the mentioned event would occur again.
You can probably get the software from HP's dowload site, or it should be on the cd's that came with the server.

Btw, this event also indirectly caused problems with our software-management-tool, Novadigm's Radia (nowadays HP's Radia), so I am wondering if any of you are using Radia as well?

[Terri Harris]

Lex,
To uninstall the Netserver agents:

1. In Windows select START - PROGRAMS

2. Look for "HP Server Agents"

3. Selecting this option will display an option to uninstall.

It is recommended to use the Netserver agent uninstall and not the Add/Remove programs from the Windows Control Panel.

[Philip Koo]

I have 2 HP LHr running Windows 2000 SP3. After applying following Microsoft security patch:
MS04-014 (KB837001)
MS04-012 (KB828741)
MS04-011 (KB835732)

Following events showed up every 30 min.

"HP Event Log : ScsiSMART: Device failed: host adapter 0, Target ID 0 and LUN 0. "

"HP Event Log : ScsiSMART: Device recovered: host adapter 0, Target ID 0 and LUN 0. "

Not sure which patch caused this. It happened on both server after the same 3 patchs are applied.

[Terri Harris]

The SCSI SMART Detail Agent is a known issue. The solution is to uninstall this agent from the Netserver agents. The steps required are:

1. Uninstall Netserver Agents: use the uninstall option from START - PROGRAMS = HP Server Agents - Uninstall.

2. Install Netserver Agents. Select the CUSTOM option. Uncheck the box for SCSI Detail Agent.

[Neil Dixon_1]

I have also been getting the same errors as Philip Koo on some of our NetServer 1000r's.
We have 95 1000r servers running Windows 2000 Server, after patching these servers we started receiving:
"HP Event Log : ScsiSMART: Device failed: host adapter 0, Target ID 0 and LUN 0. "
"HP Event Log : ScsiSMART: Device failed: host adapter 2, Target ID 0 and LUN 0. "

"HP Event Log : ScsiSMART: Device recovered: host adapter 0, Target ID 0 and LUN 0. "
"HP Event Log : ScsiSMART: Device recovered: host adapter 2, Target ID 0 and LUN 0. "

These errors have only occured since the MS04 patches were installed, and only on about 10% of the servers.

Is there a workaround for this issue, or an updated version of SCSI SMART which will correctly report on the status of the disks?

[Michel M.]

Hi,
I have noticed the same issue on our Netservers LPr (running W2000 server) as they have been fixed with the MS04 patches...
If we disable SCSI smart detail agent, how will we be informed when a real disk failures occurs ?
Is HP working on a fix ?

[Terri Harris]

The SCSI Detail Agent primarily reports the drive configuration. Disabling it does not affect the ability of TopTools to report drive problems/failures.
No further updates are expected for HP TopTools.

[Michel M.]

Thanks Terri,
The remaining problem with the removal of the agent is the required reboots... not so easy on production servers.
I have found an other solution in the HP 'toptools' forum :
- net stop SNMP
- rename or delete the file called symdb.db
(toptools device database)
- net start SNMP (and a new clean symdb.db is created)
Have a look at this link, and tell me what do you think of it :
http://forums1.itrc.hp.com/service/forums/questionanswer.do?admit=716493758+1084287454053+28353475&threadId=584711

[Terri Harris]

Michel,
I will need to test this proposed solution first. I will post here again as soon as I have more information on this.