PHCO_233261 I/O error at backup time

[Lana Cameli]

Hi there;
I use BRU utility to do my daily backup on my HP-UX v10.20 In the last couple of weeks, I have been receiving a peculiar erorr:

bru: [E018] "./new/hp/patches/PHCO_23261": read error: errno = 5, I/O error
bru: [W019] "./new/hp/patches/PHCO_23261": warning - file was truncated

I looked it up in the BRU manual which stated that the error is generated by HPUX. I went to the directory and the file is there (thouigh, admittedly, I can't tell if it's truncated or if something is using it) I'm stumped and not sure what to do at this point.

If anyone have any suggestions/recommendations on how I can resolve this, I'd really appreciate it.

[Peter Godron]

Lana,
I would suspect a disk error. Perform a dd to /dev/null to test it.
If the file was in use none of the file could be backed up.
Anything showing with: dmesg ?

[Kent Ostby · Accepted solution]

Lana --

Do a "dd" as follows:

dd if= of=/dev/null bs=64k

If it returns I/O error then one of the disks in that lvol has a hardware problem.

You can replace with each rdsk (e.g. /dev/rdsk/c0t6d0) and isolate which disk is bad then open a support case with HP hardware to have the drive replaced.

[Lana Cameli]

Thank you Peter and Kent;
It does appear that I have not one, but two bad disks. The good news, it's not my root disk. :)

I'll place a call for support to HP.

[Bill Hassell]

And to help out with Unix style error codes, look for the word "errno" which means error number. 5 is the standard code for I/O error meaning that the device (unfortunately not always stated explicitly) has returned a failure code. errno 5 is always a hardware failure of some sort, including parity or checksum errors or access errors. Access errors occur when the next record is beyond the end of the device, typically a filesystem where the underlying logical volume was reduced.

I have attached a simple script which I call errno. You give it the error code, either numeric or symbolic. errno 5 is EIO.

[Lana Cameli]

Thanks, Bill
I had a look and my logical volumes are fine, none over 50%. But I'm wondering, wouldn't fsck be able to repair the files? Outside that error in backup, there are no other symptoms showing.

[Jeff Schussele]

The only way fsck could help is if the problem was inode or meta-data. If the problem is physical - i.e. a bad spot on the disk - fsck would be of no help.

Rgds,
Jeff

[Lana Cameli]

Is there a method to determine if the error is inode or meta-data (other then fsck)?

[Jeff Schussele]

Well what I would do is run fsck with -m to see if the FS is marked clean. If not then run it with -o full and follow the output to see if it's successful. If it is then it was not a physical problem & you should be good to go.
BTW - a second fsck after an "apparent" successful cleaning is *alway* a good idea - just to be sure.

HTH,
Jeff

[Bill Hassell]

Based on the error message, it would seem that the file is corrupted rather than the directory structure. A simple cp of the file will probably terminate part way through with an I/O error. As mentioned, an I/O error cannot be fixed with fsck. It is a damaged area on the disk and while specialized hardware might be able to manipulate the disk head to recover the bad tracks but other than running mediainit (which destroys all the data), the bad spot will remain.

And one thing about bad spots...they never get better and usually precede additional bad spots or total failure of the disk. I would make an emergency backup of the problem directory by removing the file (you can always replace that patch), then run your backup and finally replace your disk. If this is your vg00 root volume, there are a lot more tasks ahead, especially if you did not mirror the root disk.

[Lana Cameli]

Thank you everyone for your help. It was, in fact, two bad disks that needed to be replaced.

I have found a fantastic paper on disk replacement in IT Resource Center and it made disk replacement a breeze. The link, for those interested, is:

https://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=926691&admit=-682735245+1148993424899+28353475

[Lana Cameli]

https://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=926691&admit=-682735245+1148993424899+28353475