ProLiant Servers (ML,DL,SL)
cancel
Showing results for 
Search instead for 
Did you mean: 

Input/output error ProLiant DL380G3

 
SOLVED
Go to solution
lastgreatone
Regular Advisor

Input/output error ProLiant DL380G3

Here is the server specs:
HP ProLiant DL380G3 (3GB RAM)
OS: RHES 2.1_2.4.9-e.65smp
2 x 72.8GB 10K U320 Hot Swap, no RAID

Server was unexpectedly powered down along with UPS, no clue what the cause was. But since then the root disk's LED turns red after approximately 1 hr after reboot. And I get Input/output errors, and dmesg shows:
kernel: cciss: cmd f69a024c has CHECK CONDITION, sense key = 0x3
Sep 12 10:40:00 digir1 kernel: EXT3-fs error (device cciss0(104,5)) in ext3_new_inode: IO

Is this a RedHat kernel issue or a hard disk issue?

17 REPLIES
HugoAlmeida
Trusted Contributor

Re: Input/output error ProLiant DL380G3

That looks like a hard disk problem to me.
cciss is the RAID driver, and it is reporting a problem, so everything points to a failed disk.

lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

OK thanks.

I'm not using RAID, should I still reconfigure the new disk with cciss?
HugoAlmeida
Trusted Contributor

Re: Input/output error ProLiant DL380G3

Sorry, cciss is the hard disk controller driver :) .
lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

what diagnostic tool should I run to confirm it's a hd problem? On HP-UX I could run ioscan but I don't know what to run on RedHat.
Torsten.
Acclaimed Contributor
Solution

Re: Input/output error ProLiant DL380G3

I guess there wasn't a proper shutdown, so the filesystem got a problem (EXT3-fs error).

I would check disks with linux exactly like with hp-ux:

dd if= of=/dev/null

if there are some I/O errors, you have a hardware problem.

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

One more question:

The command lspci shows that there is a hardware raid configured on this server, done by the vendor. I did not request it but it's there.

Is it possible the raid controller is the problem, broke due to unclean shutdown? On the front panel of the server there is an image of a disk. Soon after rebooting this image lights up with a Red X. It's not flashing. And maybe the I/O errors are due to the Raid controller trying to rebuild, but what?

What diagnostic tool can I run to test for the Raid controller?
Torsten.
Acclaimed Contributor

Re: Input/output error ProLiant DL380G3

I don't know about the red hat onboard utilities, but try to download the smart start CD image, burn it and start from

http://h18023.www1.hp.com/support/files/server/us/download/23318.html

there is a tool named ACU (array config utility) to configure/view your array/RAID.
Check out!

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
Prashant (I am Back)
Honored Contributor

Re: Input/output error ProLiant DL380G3

Hi,

Top to bottom led options :
1
Internal Health
Green = Normal Amber = System degraded. [ 1 ] Red = System critical. [ 1 ]

2
External health (Power Supply)
Red = Critical power supply failure Amber = Power redundancy failure Green = Normal

3
NIC 1 link/activity
Green = Network link Flashing = Network link/activity Off = No link/activity (if power is off, view rear panel RJ-45 led's)

4
NIC 2 link/activity

5
Front Unit Identification (UID) Switch
Blue = Activated Flashing = System remotely managed Off = Deactivated

6
Power On/Standby button & system power LED
Green = System on Amber = System shut down, but power still applied Off = Power cord not attached or power supply failure


------------------------------------------

To check the raid status and failure you can use.
ACU/ADU
avaible for linux also on site
and avaiable on SmartCd also

Regards,
Prashant s.
Nothing is impossible
lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

Do I simply run cpqacuxe from the HP SmartCD? I don't see an acu-xe rpm to install from the cd. The README.TXT makes reference on how to configure with cpqacuxe but no reference on how to troubleshoot. I'm assuming it will be apparent from the gui?
lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

Standby. I booted off the cd and am running the diags and it appears there is a POST problem.
lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

I ran the server diagnostic utility, reports following problem:

POST Device: POST Test
1727:Drive Array: New Logical
Drive(s) Attachment Detected Ran on CPU 0

Anyone know what the problem is?
lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

Here the result of compaq survey utility:

CISS Physical Drive Information
----------------------------------------

Physical Disk (Controller 0 Bus 1 SCSI ID 0):
---------------------------------------------
Drive Model: COMPAQ BD07285A25
Size: 69460.84MB
Drive SN: 3HZ104ZA00007337QM8T
Drive FW Revision: HPB3
Compaq Part: Yes
Designated as Spare: No
Drive Failed: No
Drive Rebuilding: No
Last Failure Reason: Timeout
Physical Box On Bus: 0
Physical Bay in Box: 0

Physical Disk (Controller 0 Bus 1 SCSI ID 1):
---------------------------------------------
Drive Model: COMPAQ BD07285A25
Size: 69460.84MB
Drive SN: 3HZ0K0W3000073252X5Y
Drive FW Revision: HPB3
Compaq Part: Yes
Designated as Spare: No
Drive Failed: No
Drive Rebuilding: No
Last Failure Reason: Unknown
Physical Box On Bus: 0
Physical Bay in Box: 1


Prashant (I am Back)
Honored Contributor

Re: Input/output error ProLiant DL380G3

HI,

Check cables for bent pins. If this does not resolve the issue, try "shocking" the drives.

If this does not resolve the issue, try removing ALL drives from the server, and run System Erase after verifying that all HDDs internal and external have been disconnected from this server. Then reinstall the drives, boot to a SmartStart CD, run SCU, set boot order, and reboot.

Regards,
Prashant S.


Nothing is impossible
lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

What do you mean by "shocking the drives"?
Prashant (I am Back)
Honored Contributor

Re: Input/output error ProLiant DL380G3

Hi,

This just checking the drive when it is offline.
Try to do the above and update. What is the outsome then we will what we can do.

Regards,
Prashant s.
Nothing is impossible
Prashant (I am Back)
Honored Contributor

Re: Input/output error ProLiant DL380G3

Hi,

If you see the last error reported :

-----------------------------------------
Physical Disk (Controller 0 Bus 1 SCSI ID 0):
---------------------------------------------
Drive Model: COMPAQ BD07285A25
Size: 69460.84MB
Drive SN: 3HZ104ZA00007337QM8T
Drive FW Revision: HPB3
Compaq Part: Yes
Designated as Spare: No
Drive Failed: No
Drive Rebuilding: No
Last Failure Reason: Timeout
-----------------------------------------
It is timeout so i told you to try same.

Regards,
Prashant S.
Nothing is impossible
lastgreatone
Regular Advisor

Re: Input/output error ProLiant DL380G3

I installed a new drive and I have a healthy system again. thanks.