Document: "When Good Disks go Bad ..."

[LVM Support Team]

White Paper Posted on docs.hp.com:
"When Good Disks go Bad
Dealing with Disk Failures under LVM"


Abstract:

This white paper discusses how to deal with disk failures under HP-UX's Logical Volume Manager (LVM). Targeted at the System Aministrator or Operator who has experience with LVM, it includes strategies for preparing for disk failure, means for recognizing that a disk has failed and steps for removing or replacing a failed disk.

Details:

LVM is hugely popular and is used on 96% of all HP-UX installations. LVM is also a complex software product. One of the most complex LVM related tasks is dealing with failed disk drives. However, armed with proper knowledge and preparation, the impact of a failed disk can be greatly minimized. This white paper details the following topics (including mirroring):
- How to Prepare for Disk Recovery
- How to Recognize a Failing Disk
- How to Confirming a Disk Failure
- How to Choosing a Course of Action
- How to Remove a Failed Disk
- How to Replace a Failed Disk

The paper can be accessed at:
http://docs.hp.com/en/5991-1236/When_Good_Disks_Go_Bad.pdf

After reading the white paper, please take a few moments to answer some questions so that we may improve the content of the paper.


Thank You,
Hewlett Packard LVM Support Team
------
Survey questions:

Please rate the following questions on a scale of 1-10
( 1 is the least and 10 is the most)
- The information is technically correct?
- The information presented helps my understanding of LVM?
- I will be/have been able to use the proceedures provided?
- The proceedures were easy to follow?

Please give us your Thoughts:
- Would you recommend that others read this paper? (Why or why not)
- Are there any additional LVM topics for which you would like more information?

[Pete Randall · Accepted solution]

9 - The information is technically correct?
9 - The information presented helps my understanding of LVM?
9 - I will be/have been able to use the proceedures provided?
9 - The proceedures were easy to follow?

Please give us your Thoughts:
- Would you recommend that others read this paper? (Why or why not)

Yes, it's concise, complete and logical. It also covers Itanium architecture in one source, making it concise, complete, compact, and convenient as well as logical!


- Are there any additional LVM topics for which you would like more information?

Not off the top of my head.


Pete

[Devender Khatana]

Hi,

10 - The information is technically correct.
9 - The Information presented helps my understanding of LVM.
8 - I will be/have been able to use the proceedures provided.
9 - The proceedures were easy to follow.

Yes would always recommend this paper to everybody who seems to be in trouble in failed disk scenarios.

Important - This document could have been even more worthful if it could have included details of limits of various LVM parameters & there effect when the failed disks are replaced with different capacity disks.

Regards,
Devender

[DCE]

Please rate the following questions on a scale of 1-10
( 1 is the least and 10 is the most)
- The information is technically correct?
10

- The information presented helps my understanding of LVM?
10

- I will be/have been able to use the proceedures provided?
10

- The proceedures were easy to follow?
10

Please give us your Thoughts:
- Would you recommend that others read this paper? (Why or why not)

Yes. This document represents a "single source " place to obtain information necessary to maintain and repair LVM based disks.

- Are there any additional LVM topics for which you would like more information?

[Robert Bennett_3]

Great Paper!

10 - The information is technically correct.
10 - The information presented helps my understanding of LVM.
10 - I will be able to use the procedures provided.
10 - The procedures are easy to follow.

I have already sent the link to fellow employees.

The only thing I can think of is an appendix on how to handle the disposal of failed disks - i.e. removal of data best procedures. This may be a good paper in which to address this topic. It's not LVM, but it does go hand-in-hand with disk replacement.

Thanks for the great work!

[Mel Burslan]

Please rate the following questions on a scale of 1-10
( 1 is the least and 10 is the most)
10- The information is technically correct?
9- The information presented helps my understanding of LVM?
10- I will be/have been able to use the proceedures provided?
10- The proceedures were easy to follow?

Please give us your Thoughts:
- Would you recommend that others read this paper? (Why or why not)

Yes it will be recommended. It covers the topics in good detail and at the same time, understandable by admins of junior level experience.

- Are there any additional LVM topics for which you would like more information?

Not really but again, it has been a while since I haven't used any HP/UX documentation at all.

[Devesh Pant_1]

I say this is a great paper. It helps the Systems Administrator in many ways. A must have document for the SA.
10 - The information is technically correct.
10 - The information presented helps my understanding of LVM.
10 - I will be able to use the procedures provided.
10 - The procedures are easy to follow.

[ralph barber]

Very useful thread however under
â ¢ Maintain adequate documentation of your I/O and LVM configuration, specifically outputs from these commands:

Shouldn't it be vgcfgbackup rather than vgcfgrestore ? Or am I missing the point her

[ralph barber]

To answer my own question
Yes I was missing the point the -l shows what you have available from backup configs
I should look before I leap !!!

[Mahesh Kumar Malik]

Hi LVM support team

10- How to Prepare for Disk Recovery
9- How to Recognize a Failing Disk
9- How to Confirming a Disk Failure
9- How to Choosing a Course of Action
9- How to Remove a Failed Disk
9- How to Replace a Failed Disk

Please keep us posted on future updates on same subject

Regards
Mahesh

[Thayanidhi]

Hi LVM team,
9 The information is technically correct?
9 The information presented helps my understanding of LVM?
9 I will be/have been able to use the proceedures provided?
9 The proceedures were easy to follow?

Would you recommend that others read this paper? (Why or why not)
Yes definitely, to colleagues and even customers. Helps them to understand.
Are there any additional LVM topics for which you would like more information?

Request to post any deleopments in the same location (docs.hp.com)

Regds
TT

[Bill Hassell]

A couple of problems:

- dump_lvmtab is not available on HP media or from HP websites. I suggest it be put onto hprc.external.hp.com, the Response Center's file distribution site.

- One the last page there is a feedback link for docs.hp.com that is badly broken. It reports:

"The plug-in required by this 'URI' action is not available..."

This is actually a PDF error message from Acrobat Reader and only occurs when you click on the link in the PDF document. Typing the link in by hand works OK.

[Eknath]

Hi,
Thank you very much for this information. It clarifies many doubts. I am passing the information to other sys. Admins

Here are my ratings
( 1 is the least and 10 is the most)
10- The information is technically correct?
10- The information presented helps my understanding of LVM?
10- I will be/have been able to use the proceedures provided?
10- The proceedures were easy to follow?

Thanks once again
eknath

[Uwe Zessin]

I can't comment on the technical correctness, but it certainly looks useful to me, thanks! I'll forward it to my colleagues.

One minor comment: as far as I can tell - EMS is not supported for/ does not work on the storage arrays (EVA, HSG) that came from the Compaq line. I think this should be mentioned.

[Adisuria Wangsadinata_1]

Hi LVM Support Team,

It's cool ... will foward this information for my team. Thanks for your help.

Please rate the following questions on a scale of 1-10
( 1 is the least and 10 is the most)
10 - The information is technically correct?
10 - The information presented helps my understanding of LVM?
10 - I will be/have been able to use the proceedures provided?
10 - The proceedures were easy to follow?

Cheers,
AW

[Babu A]

Hi LVM Support Team,

This doc is very much usefull. We can able understand fully.

10 - The information is technically correct?
9 - The information presented helps my understanding of LVM?
10 - I will be/have been able to use the proceedures provided?
10 - The proceedures were easy to follow?

Thanks a lot making this thread.

Regards,

Babu

[Sudeesh]

Amazing !!! its really useful.


Please rate the following questions on a scale of 1-10
( 1 is the least and 10 is the most)

10 - The information is technically correct?
9 - The information presented helps my understanding of LVM?
10 - I will be/have been able to use the proceedures provided?
10 - The proceedures were easy to follow?

Please give us your Thoughts:
- Would you recommend that others read this paper? (Why or why not)

Yes. This something a sys admin must have with him.

- Are there any additional LVM topics for which you would like more information?

I would like to know more about LVM OLR



Thanks

Sudeesh

[Siju Jose_1]

The information is very much useful,thanx

9- The information is technically correct?
9- The information presented helps my understanding of LVM?
9- I will be/have been able to use the proceedures provided?
9- The proceedures were easy to follow?


- Would you recommend that others read this paper? (Why or why not)

Already gve the link to team so that everybody reads it

[Brian Butscher]

What good timing...
I just lost a disk in vg00. I saw this paper and decided to try using the steps to remove and reinstall a disk. I had to forcibly reduce vg00 to remove the disk from vg00 with vgreduce -f vg00, I moved /etc/lvmtab to /etc/lvmtab.save and then performed a vgscan -v and vg00 didn't show up in lvmtab. This is a hot-swap disk so I swapped out the defective disk with a known good disk.
I'm still working on getting vg00 into lvmtab.

Please rate the following questions on a scale of 1-10
( 1 is the least and 10 is the most)
9 - The information is technically correct?
9 - The information presented helps my understanding of LVM?
8 - I will be/have been able to use the proceedures provided?
9 - The proceedures were easy to follow?

I would recommend this paper to others.

Regards,

Brian

[TwoProc]

- The information is technically correct?
9
- The information presented helps my understanding of LVM?
9
- I will be/have been able to use the proceedures provided?
10
- The proceedures were easy to follow?
10

Please give us your Thoughts:
- Would you recommend that others read this paper? (Why or why not)
Yes.
- Are there any additional LVM topics for which you would like more information?
Cover aspects use of PVGs.

Comment: I liked the part of how it showed how to read the device number from the lbolt statement in the syslog. Didn't know that!

[David Child_1]

What horrible timing :), I just got done replacing a failed disk about an hour ago. I didn't run into any major problems, but I was unaware of the new pvchange -a option which would have made it a tad easier.

10 - The information is technically correct?
9 - The information presented helps my understanding of LVM?
10 - I will be/have been able to use the proceedures provided?
10 - The proceedures were easy to follow?

Please give us your Thoughts:
- Would you recommend that others read this paper? Definately. Even if you are experienced you might run across something new.
- Are there any additional LVM topics for which you would like more information? I cannot think of anything at this time.

Thanks,
David

[Henk Geurts]

very complete document, you even mentioned the removal of "ghost-disks" with the pv key. good work..

10 - The information is technically correct?
9 - The information presented helps my understanding of LVM?
10 - I will be/have been able to use the proceedures provided?
9 - The proceedures were easy to follow?

Please give us your Thoughts:
- Would you recommend that others read this paper? yes, i will send the link to all HPUX-admins and engineers in my company.
- Are there any additional LVM topics for which you would like more information? none i can think of.

Thanks!

[Deoncia Grayson_1]

Please rate the following questions on a scale of 1-10
( 1 is the least and 10 is the most)
The information is technically correct? 9
The information presented helps my understanding of LVM? 9
I will be/have been able to use the proceedures provided? 9
The proceedures were easy to follow? 10

Please give us your Thoughts:
- Would you recommend that others read this paper? I probably will, very valuable information.

Are there any additional LVM topics for which you would like more information? None at the moment

[Jan van den Ende]

Hi.

Sorry, no comment as to the accuracy of the article, as it is out of my area of expertise.

And this is exactly my criticism:
the title "When Good Disks Go Bad" promises much more than the article contains.

For instance, it does not deal with HBVS sets, or MSCP, nor with any other technology which I also do not know about.

Now, if the title would have been:
"When good LVM disks go bad", that probably would have been a more direct pointer for the intended audience, and I would not have bothered.
Maybe from your perspective all disks are LVM disks, but really, the world is wider than that!

hth,

Proost.

Have one on me.

jpe

[generic_1]

I think you did a nice job with the document. You may want to consider adding san topics and commands to this. If you have multiple storage arrays tracking down disks, and failed fibercards are good to mention. Utilities like tdutil and fcms util could be handy when you have disks go offline :) which could mimic a disk failure.
Tracking down lun
Diskreplacement for wwn ect.