topic Re: System Warning in Disk Enclosures

System Warning

Chartier Jerome — Wed, 15 Jan 2003 09:49:04 GMT

Hello everyone,

After an unvoluntary power cut, I have "system warning" on my Disk array led display, and the syslog has entries:Jan 14 01:46:33 manon vmunix: gate64: sysvec_vaddr = 0xc0002000 for 1 pages
Jan 14 01:46:33 manon vmunix: Unexpected interrupt on EIRR bit 32
Jan 14 01:46:33 manon vmunix: NOTICE: autofs_link(): File system was registered at index 3.
Jan 14 01:46:33 manon vmunix: NOTICE: nfs3_link(): File system was registered at index 5.

Jan 14 01:48:57 manon EMS [1133]: ----- EMS Monitor Restart due to system reboot ----- Title: disk_em Command: /usr/sbin/stm
/uut/bin/tools/monitor/disk_em Vendor: Hewlett-Packard Company Version: A.01.00 To obtain a list of currently monitored
resources, execute the following: /opt/resmon/bin/resdata -M 1533300606
Jan 14 01:49:34 manon EMS [1143]: ----- EMS Monitor Restart due to system reboot ----- Title: FC60_monitor Command: /usr/sbi
n/stm/uut/bin/tools/monitor/fc60mon Vendor: Hewlett-Packard Co. Version: A.01.05 To obtain a list of currently monitored
resources, execute the following: /opt/resmon/bin/resdata -M 1652016795
Jan 14 01:51:04 manon EMS [1250]: ------ EMS Event Notification ------ Value: "MAJORWARNING (3)" for Resource: "/storage/events/
disk_arrays/AutoRAID/000000105241" (Threshold: >= " 3") Execute the following command to obtain event details: /opt/resm
on/bin/resdata -R 81920018 -r /storage/events/disk_arrays/AutoRAID/000000105241 -n 81920001 -a
Jan 14 01:51:05 manon EMS [1250]: ------ EMS Event Notification ------ Value: "MAJORWARNING (3)" for Resource: "/storage/events/
disk_arrays/AutoRAID/000000105241" (Threshold: >= " 3") Execute the following command to obtain event details: /opt/resm
on/bin/resdata -R 81920018 -r /storage/events/disk_arrays/AutoRAID/000000105241 -n 81920002 -a
Jan 14 03:18:22 manon syslog: HP Advanced Disk Array serial number 000000105241: Multi-initiator lock obtained: Product Revision:
HP56: Product ID: C5447A
Plus, one disk in the raid array has an orange fixed led, and one array controller has a fixed green led (no more flashing).

In my opinion, i've got an array+disk trouble issue.

Is there any more test to perform?
Have you got an idea?

Regards

Thanks in adavnce for your help

J??r??me

Re: System Warning

Arturo Perez del Galleg — Wed, 15 Jan 2003 10:43:23 GMT

Hi Jerome!
The first messages are messages about the reboot of the service (EMS), but are you test the MAJORWARNING message, specially the contents of the pointed command?
/opt/resm
on/bin/resdata -R 81920018 -r /storage/events/disk_arrays/AutoRAID/000000105241 -n 81920001 -a
HTH

Re: System Warning

Chartier Jerome — Wed, 15 Jan 2003 10:48:21 GMT

Yes, I've got this log:ARCHIVED MONITOR DATA:

Identification

Array S/N: 000000105241, Path: /dev/rdsk/c5t10d0, Path: /dev/rdsk/c6t11d0

Summary:

Data Redundancy has been lost.

Missing Disk: SEAGATE ST118202LC ## 869600001940

Description of Event:

This event message is displayed when there is not sufficient

information available to reconstruct user data should another disk

fail or be removed from the volume set.

'Data Redundancy Loss' represents a degraded state, whereas, 'Data

Unavailability' represents a total lack of ability to access user

data. The 'Data Redundancy Loss' condition takes into account disk

drives only, and not other components such as controllers, fans, etc.

Thus, the failure of just one fan or controller in a configuration

with redundant components does NOT cause 'Data Redundancy Loss.'

The only cause is one disk from the volume set is inaccessible.

Probable Cause / Recommended Action:

Perform a rebuild on the disk array. You may need to add more disks.

I've performed then a rebuild manually on the console, but in the view settings, he can see just 9 disks( I've got 10 disks in the disk array).

Any idea?

Thanks in advance

J??r??me

Re: System Warning

Eugeny Brychkov — Wed, 15 Jan 2003 12:15:05 GMT

This is an autoraid issue. Please run the following commands:
'arraydsp -a 000000105241' = array state
'logprint -v 000000105241' = logs from controllers
Please zip these outputs and attach to your next reply. After reviewing it I will be able to tell you which disk is dead and what to do. I guess you can open autoraid's front door and locate failed disk shown by yellow disk led
Eugeny

Re: System Warning

Chartier Jerome — Wed, 15 Jan 2003 12:30:12 GMT

here is the log

thanks

Jerome

Re: System Warning

Eugeny Brychkov — Wed, 15 Jan 2003 12:50:14 GMT

Logprint output if not complete. Only for today. I see that array starts to rebuild sometimes, and disk A6 was removed and reinstalled (at the same second?). the same looks like for A5.
Recommendations:
1. check if these drives, A5 and A6, are installed securely into slots;
2. call HP and order autoraid firmware upgrade (to HP62) and its disks firmware upgrade;
3. watch for logprn output if there will be more events like 'disk failed', 'disk removed', 'rebuild started/completed' indicating that something wrong is happening to the array.
Please note that you're allowed to remove only one disk at a time and wait for rebuild to complete, otherwise data loss will occur. The same for disk failure: only one disk can fail at one time until rebuild will be completed.
Id you wish you can attach whole logprint output (see 'man logprint') zipped
Eugeny