Disk Failure

[Jim Mallett]

ISEE reported an "unrecovered read error" on one of my disks, which happens to be part of vg00. I ran a dd against it and it failed about a minute into it. Does this necessarily mean a pvmove would fail? I was thinking about a pvmove, replace/rebuild the boot disk, pvmove the data back.

This is a K580 with 4 2G internal drives for VG00 (no mirroring). I would love to find a way to take care of it without having to Ignite it (not literally).

pvdisplay /dev/dsk/c0t3d0 reports the following lv's exist on it:
/dev/vg00/swapnew
/dev/vg00/lvol5 (/var)

lvlnboot reports that this disk is the boot disk and it contains root, swap, and dump:
/dev/dsk/c0t3d0 (10/0.3.0) -- Boot Disk
/dev/dsk/c0t4d0 (10/0.4.0)
/dev/dsk/c0t5d0 (10/0.5.0)
/dev/dsk/c0t6d0 (10/0.6.0)
Root: lvol1 on: /dev/dsk/c0t3d0
Swap: swapnew on: /dev/dsk/c0t3d0
Dump: swapnew on: /dev/dsk/c0t3d0, 0

Should I forget about the pvmove and just face the facts that I will need to shutdown, replace disk, and Ignite?

Thanks for any ideas....
Jim

[Pete Randall]

Jim,

A pvmove *MIGHT* work, but I think it's unlikely. If dd failed, I would expect that your pvmove is going to have similar issues and Ignite is going to be your ultimate saviour.

That said, if you have the spare disk, it couldn't hurt to try the pvmove and see what happens!


Pete

[Patrick Wallek]

Hmm....A 'dd' failure wouldn't necessarily indicate that pvmove would fail. The spot that the dd failed on could be anywhere. The failure of the 'pvmove' would depend on whether or not that particular spot on the disk actually has data.

The "unrecovered read error" could be a bit more worrisome. Something tried to read something on the disk and failed. I would take that more into account for a potential pvmove failure.

Before totally throwing in the towel, try a new Ignite backup of this machine and see if it succeeds or if you get an 'unable to read file' type error somewhere along the line.

Do you really need the swap area on this disk? Could you comment the swap out in /etc/fstab and reboot and see if you get the error again? You could then try to move /var to another disk if you have room. If you need that swap you could also try recreating it on a different disk.

This is why I LOVE mirroring.

[Sanjay_6 · Accepted solution]

Is it possible to do mirroring even at this time. If so that would be the way to go. Doing it any other way is going to be a difficult situation. You may have to do a clean reinstall / reignite.

Hope this helps.

Regds

[RAC_1]

While there is no harm in trying pvmove, I would also try to do follows, if data is very important.

If ca into single user mode, mount the said lvolv(pvdisplay to know it) in read only and back it up and then newfs on lvol and then restore. newfs would mark that block as bad block.

But always better to replace the disk.

Anil

[Jim Mallett]

I'm starting the Ignite now, just in case. If it doesn't report a read error I'll try the pvmove.

I have 2 18G disks onsite that I could put in the K580, but there are only 4 internal bays so I would have to free 2 of them up. If I could get the 2 18G disks in there it would solve a few problems:
- I'd be able to mirror.
- The 2G disks are going off support in 3 months.

Now the trick is, accomplishing this in a 1 hour window on Sunday morning. I'm going to go out kicking and screaming before I have downtime on this box, it's been far to long (and yes, I've been far too lucky considering vg00 isn't mirrored).

I'll post the outcome....any more ideas are appreciated though.

Jim

[Patrick Wallek]

What kind of data disks do you have on this box? Any way you could mirror to external drives, reduce the internals, shut down, put your 18GB in, then mirror back to them?

[Jim Mallett]

I have an Model 30 array fiber connected. I also have a Model 20 array (scsi) that I took offline a while ago. I also have the option of using Symmetrix disk (fiber).

That's one idea that hadn't even crossed my mind.

The Ignite just finished. It didn't fail, just finished with warnings. Now this is strange, the two files it states it's getting the read error on are two files that I know I deleted on Friday, and confirmed.

Jim