vxfs: mesg 056: vx_dataioerr, but no apparent disk error

[Dag A.]

Hi.
I have a HP-UX machine which seems to have a corrupt file, although I cannot find any disk errors:

System: HP-UX B2600 ver 10.20
Disk: SCSI internal, 73GB

I get the following messages on startup:

1. syslog.log:
vxfs: mesg 056: vx_dataioerr - /dev/vg00/lvol5 file system file data read error

2. Oracle, alert_advantdb.log:
ORA-01115: IO error reading block from file 7 (block # 37)

3. The database file in question shows up with status AVAILABLE in Oracle, and there seems to be no problems with the tablespace.
(this db file is on the volume in question, lvol5)

4. Cannot run diskinfo, I get the message:
"Character device required"

5. lvdisplay shows all logical extents as "current" (ie no stale extents)

6. pvdisplay shows all physical extents as current or free

7. fsck -m shows status OK for lvol5

8. There is only one disk, and the system boots OK until Oracle starts.

I'm a bit puzzled by this, hope that there is someone that have some ideas.

Best regards

[Patrick Wallek]

This sounds more like a file-system corruption problem rather than a disk problem.

4. Run diskinfo with the /dev/rdsk/c?t?d? for the appropriate disk device, though I don't think this will tell us much.

7. If possible run a 'fsck -o full' on the LV and see what it reports.

8. "....system boots OK until Oracle starts."

I'm not sure I understand this. Is Oracle up and running or not?

If you have a relatively current backup, I would say it is time to use it. You can try restoring only the effected data file, but if there is file system corruption, there could be other things that are a problem as well.

Is the Oracle data really in VG00/lvol5? That is /home on my systems? That's a bad place for Oracle data.

My suggestion was going to be to recreate the filesystem from scratch and restore, until I noticed that it is vg00/lvol5.

[Dag A.]

Hi.
Thanks for your answer.

Oracle is running, and I can connect with sqlplus to the oracle schema which uses db file #7. All the tables are present.
(Yes, the db file is on the /home volume, don't know why it was done that way)

This situation occured after a full backup was run and the machine was restarting. It remains to see if there is an also errors in the file on tape. Maybe I have to use an older backup.

I did run diskinfo with /dev/rdsk/c2t0d6 option when it complained about "Character device required"

[cnb]

See this article:

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=120&prodSeriesId=3215383&prodTypeId=18964&objectID=c00992613

Rgds,

Please remember to assign points to those who help you...
http://forums11.itrc.hp.com/service/forums/helptips.do?#33

;-)

[Dag A.]

Hi.
I did unmount the /home volume and ran the fsck -m command, it reports "OK" on lvol5. Would it do that if there was a hardware error? Anyway, I guess that if there is a hardware error, fsck will not be able to fix it.

I will probably need to restore the whole file system from backup, as there will probably be some conflicts if I only restore the DB file.
The question is whether I should replace the harddisk first, if there really is a HW error

[ankitj1983]

The main causes for an ORA-01115 are:

1. HARDWARE PROBLEMS
- Disk controller problems: the most common, and usually intermittent.
- Disk problems: these include bad blocks, disk malfunctioning, etc.

2. DATA BLOCK CORRUPTION (AT THE PHYSICAL LEVEL)

Usually caused by previous hardware problems.

3. PROBLEMS HANDLING VERY LARGE DATAFILES

In the same note it suggests to check the objects involved.
"If the datafile belongs to a user or index tablespace, you may also
address the problem as an object recreation issue if the ORA-01115
occurs consistently against the same objects (tables, indexes, etc.).
The following query returns the object in which the bad block is: "

SELECT SEGMENT_NAME, SEGMENT_TYPE FROM DBA_EXTENTS
WHERE FILE_ID = 4 and 8713 BETWEEN BLOCK_ID
AND BLOCK_ID + BLOCKS - 1;

SELECT SEGMENT_NAME, SEGMENT_TYPE FROM DBA_EXTENTS
WHERE FILE_ID = 20 and 154540 BETWEEN BLOCK_ID
AND BLOCK_ID + BLOCKS - 1;

It will be a good idea to check your alertlog and relevant trace files for more information.


Regards

Ankit

[Dag A.]

Still not sure what really was the error here, but have now replaced the hard disk and done recovery from backup.
Thanks for your help.