Disk failure - "fsck: cannot initialize aggregate"

[Chris Bensend]

Hey folks,

We had a disk failure on an HP-UX 11.00 machine yesterday. The disk is an external 36G drive.

The disk contained several home directories that I'd really really like to recover (please don't ask about backups, I know, I know), but it seems to have failed in a way I haven't seen before.

Upon boot, the machine fails to mount the disk, complaining about "no such device". I attribute that to the fstab likely being out of date. Doing an ioscan reveals the following two disks:
10/0/15/1.5 target
10/0/15/1.5.0 disk SEAGATE ST336704LC
10/0/15/1.6 target
10/0/15/1.6.0 disk SEAGATE ST336704LC

Taking a look in /dev/dsk, I see devices for c3t5d0 and c3t6d0. c3t6d0 is vg00, so I assume the external drive is c3t5d0.

So, I attempt to fsck it manually:

[44] % fsck -y -F vxfs -o full /dev/rdsk/c3t5d0
invalid super-block
search for auxiliary super-block? (ynq)y
alternate super-block not found
vxfs fsck: cannot initialize aggregate
file system check failure, aborting ...

Not good. I'd appreciate any help any of you can provide - SAM considers it "unused", and nothing I've tried has worked so far.

Thanks a BUNCH!

Benny

[Eugeny Brychkov]

Benny,
as usual:
1. 'ioscan -fn' and check if disk is CLAIMED and DEVICE;
2. 'diskinfo -v' to see if disk reports correct block size, number of block, its capacity and vendor/serial number strings;
3. dd from the disk to /dev/null device using bs=4096k. This way you'll check surface for read operation.
If all these will be successful then you simply have data corrupt on the disk. Go to Vogon Intl. at http://www.vogon-international.com
Eugeny

[Patrick Wallek]

YOu really need to know what Volume Group and Logical Volume(s) were on that disk.

First do an 'ioscan -fnC disk' to see the disks and their device files.

Then do a 'strings /etc/lvmtab' to see which VG that disk belonged to. You could also try doing a 'pvdisplay /dev/dsk/c?t?d0' to see what VGs and LVs were on that disk.

Once you know what the VG name is, do a 'vgchange -a y vg??' to activate the VG. Then try doing a 'fsck -F vxfs -o full -y /dev/vg??/rlovl?'.

You can then try to mount the LV 'mount /dev/vg??/lvol? /mountpointname'.

[Chris Bensend]

Thanks for the quick replies...

The ioscan and diskinfo seemed to display good info - it is marked as CLAIMED in the ioscan, and the diskinfo reports the correct parameters.

This disk wasn't part of a volume group - it was simply newfs'ed and mounted, so the LVM stuff won't help.

Are there further steps I can take to try to recover the data (short of the data recovery service - the budget is VERY tight, although we'll have to if I can't do it)?

Thanks!

Benny

[S.K. Chan]

Is this disk (c3t5d0) a LVM disk or not ? If it is your fsck syntax should really be checking the logical volume instead of the physical disk. For example (assuming c3t5d0 is in vg01)..
# fsck -y -F vxfs -o full /dev/vg01/lvol1
If it's a whole disk configuration (not LVM) then the fsck syntax would be.
# fsck -y -F hfs /dev/rdsk/c3t5d0
What's in /etc/lvmtab ?
# strings /etc/lvmtab

[John Dvorchak]

I suppose that if you had another 36gb disk that you could newfs and mount that possibly you could dd the data from the failed disk to the new disk.

I am a little fuzzy here though. You said it "failed" and now it won't fsck the /dev/rdsk or mount it. You also said that /etc/fstab is "out of date"? I would suggest that you try to mount it the old fashioned way to see if there is anything you can do with the disk:

mount /dev/dsk/c3t5d0 /tmp_mnt

See what error, if any is generated. Notice I did not put a -F or any other switches, I am just curious what errors it generates.

No matter what happens with this test, I would suggest that you make your fstab file "Up to date".

[Michael Steele_2]

You've got absolutely no LVM at all? So a vgcfgrestore won't help.

# vgfcgrestore -n /dev/rdsk/
# vgchange -a y /dev/vgname

What capcity was this 36gig disk at before this problem? Create another file system of equvivalent size and try a low level backup with dd. First, see what's there:

dd if=/dev/rdsk/c3t5d0 of=/tmp/file bs=2048 count=1000

Then, if happy:

dd if=/dev/rdsk/c3t5d0 of=/tmp/file bs=2048

[Jon Finley]

Are you sure that this disk was vxfs? The error you're getting is the same error as if you ran fsck on a disk that had hsf on it.

Jon

[Jon Finley]

Well.. I tried to type hfs. :-(

Jon

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

Hi Benny

# dd if=/dev/rdsk/c3t5d0 of=/dev/null bs=1024k

if you get any I/O error then it's confirmed that disk failure.

[Ravi_8]

All

my sincere apologies for multiple replies as i had problem with proxy

[Michael Steele_2]

Hey Ravi:

Did you ever see that commercial where the irate user / father is trying to kill all the pop up windows from his ISP?

[Michael Steele_2]

If the prompt doesn't come back after the dd command this is also indicative of a bad disk.

[Chris Bensend]

Hey folks,

OK. I very much appreciate all the replies, and I very much regret to inform you that my problem report in the first place was flawed.

c3t5d0 ISN'T the failed disk. I should have noticed this - the disk that's bad is an external, and c3t5d0 shared the same controller as the internal boot disk?!? No - c3t5d0 is a second internal disk that I had no idea was there, that is not being used.

:(

Which means, of course, I'm a potatohead and I've been trying to fix the wrong disk. The disk that _is_ failed isn't showing up at all. Not via SEArch on boot-up, not via ioscan or diskinfo or SAM. It's just gone.

After swapping machines, cables, and enclosures, I'm pretty sure the disk is gone. Nothing we've tried will even get a machine to see the disk even being present. :(

So, thanks everyone - it looks like this one's a wash.

Benny

[Sorrel G. Jakins]

Hey Ravi

Now I know why you have a crown and I don't have anything!

Sorrel