VG device file's minor alleged to be in use

[Ralph Grothe]

Hi,

WARNING -
I have been experiencing very annoying servicing problems of ITRC forums for the last week or so.
So I have some concern whether I will be able to read this thread after having submitted it at all.
They must have refurbished the website which had the effect of making most of it now inaccessible to my user agent (i.e. Netscape 4.75, cannot get a more recent binary for AIX, which is the OS of our login server :-(
When I follow a thread, all I get serviced/displayed are some buttons but no content whatsoever.
Thus I might not be able to give feedback as quickly as I wished.

Now to my subject.
I've been asked by my admin colleague to check the LVM consistency with regards to service guard on his cluster servers, since he had been doing quite some disk reconstruction work there lately.

The first thing I stumbled over was that vg00 denied vgdisplay because of supposed minor No. collision with other VGs' group devices.


# vgdisplay vg00
vgdisplay: Couldn't query volume group "vg00".
Possible error in the Volume Group minor number; Please check and make sure the
group minor number is unique.
vgdisplay: Cannot display volume group "vg00".



Hm, that's really strange because I cannot confirm this by stat'ing the device files (please, don't ask me why the gap between 0x1b000 and 0x1e0000)


# ll /dev/vg*/group|sort -n +6
crw-r----- 1 root sys 64 0x000000 Oct 19 1999 /dev/vg00/group
crw-r--r-- 1 root sys 64 0x010000 Nov 2 2000 /dev/vg01/group
crw-r--r-- 1 root sys 64 0x020000 Sep 4 14:55 /dev/vg02/group
crw-r--r-- 1 root sys 64 0x030000 Jun 13 2002 /dev/vg03/group
crw-r--r-- 1 root sys 64 0x040000 Nov 13 2000 /dev/vg25n/group
crw-r--r-- 1 root sys 64 0x050000 Sep 10 15:55 /dev/vg31n/group
crw-r--r-- 1 root sys 64 0x060000 Nov 13 2000 /dev/vg32n/group
crw-r--r-- 1 root sys 64 0x070000 Sep 10 15:57 /dev/vg33n/group
crw-r--r-- 1 root sys 64 0x080000 Nov 13 2000 /dev/vg34n/group
crw-r--r-- 1 root sys 64 0x090000 Sep 16 14:47 /dev/vg35n/group
crw-r--r-- 1 root sys 64 0x0a0000 Nov 13 2000 /dev/vg36n/group
crw-r--r-- 1 root sys 64 0x0b0000 Nov 13 2000 /dev/vg37n/group
crw-r--r-- 1 root sys 64 0x0c0000 Sep 16 14:47 /dev/vg38n/group
crw-r--r-- 1 root sys 64 0x0d0000 Nov 13 2000 /dev/vg39n/group
crw-r--r-- 1 root sys 64 0x0e0000 Nov 13 2000 /dev/vg40n/group
crw-r--r-- 1 root sys 64 0x0f0000 Sep 18 15:45 /dev/vg41n/group
crw-r--r-- 1 root sys 64 0x100000 Sep 29 09:15 /dev/vg42n/group
crw-r--r-- 1 root sys 64 0x110000 Sep 11 15:04 /dev/vg04/group
crw-r--r-- 1 root sys 64 0x120000 Feb 7 2001 /dev/vg12n/group
crw-r--r-- 1 root sys 64 0x130000 Jul 31 10:36 /dev/vg11n/group
crw-r--r-- 1 root sys 64 0x140000 Nov 15 2002 /dev/vg30/group
crw-r--r-- 1 root sys 64 0x150000 Jul 19 2002 /dev/vg9tmp/group
crw-r--r-- 1 root sys 64 0x160000 Jul 15 10:36 /dev/vg05/group
crw-r--r-- 1 root sys 64 0x170000 Sep 19 14:31 /dev/vg05d/group
crw-r--r-- 1 root sys 64 0x180000 Sep 28 12:19 /dev/vg11d/group
crw-r--r-- 1 root sys 64 0x190000 Sep 22 14:55 /dev/vg25d/group
crw-r--r-- 1 root sys 64 0x1b0000 Sep 23 09:23 /dev/vg12d/group
crw-r--r-- 1 root sys 64 0x1e0000 Sep 27 19:10 /dev/vg30d/group
crw-r--r-- 1 root sys 64 0x2b0000 Sep 19 2001 /dev/vg50/group
crw-rw-rw- 1 root sys 64 0x1a0000 Sep 17 12:36 /dev/vg50d/group


This crashes also


# lvlnboot -v vg00
lvlnboot: Couldn't query volume group "/dev/vg00".
Possible error in the Volume Group minor number; Please check and make sure the
group minor number is unique.


But at least I can retrieve the backed up LVM data of this vg

# vgcfgrestore -f /etc/lvmconf/vg00.conf -l
Volume Group Configuration information in "/etc/lvmconf/vg00.conf"
VG Name /dev/vg00
---- Physical volumes : 2 ----
/dev/rdsk/c1t6d0 (Bootable)
/dev/rdsk/c2t6d0 (Bootable)


which matches this


# strings /etc/lvmtab|head -5
/dev/vg00
%d"88
/dev/dsk/c1t6d0
/dev/dsk/c2t6d0


Looks to me that my fellow admin's messing with the disks has somehow created yet another wee kernel vs. user space LVM inconsistency.

Any suggestions?

Rgds.
Ralph

[Ralph Grothe]

Ouch,
I can see my posting, but either the webserver has sent me or my user agent has displayed a whole mess.
No formatting or aiding whitespace to support readebility.
I hope you don't get the same clutter.

[Ralph Grothe]

Nay, now I only can see the buttons again...

HELP!

[Tom Geudens]

Hi Ralph,
You could try the following to see what vgscan suggests ...
#vgscan -p

Regards,
Tom Geudens

[RolandH]

Hi Ralph,

try a ll /dev/*/group.

Perhaps you have other places then in vg* dir, where a group file is stored.

Roland

[Michael Steele_2]

Regarding:

# strings /etc/lvmtab|head -5
/dev/vg00
%d"88
/dev/dsk/c1t6d0
/dev/dsk/c2t6d0


The "...%d"88..." is normal. I've read where its the encrypted vgid and know that it is suppose to be there.
##########################################
#
##########################################

Regarding:

"...crw-r----- 1 root sys 64 0x000000 Oct 19 1999 /dev/vg00/group..."

Shoud the permission be the same as the others? "...crw-r--r---..."
############################################
#
############################################
Also refer to this patch: PHCO_21630

Referenced in this thread:
http://forums1.itrc.hp.com/service/forums/parseCurl.do?CURL=%2Fcm%2FQuestionAnswer%2F1%2C%2C0xad338f960573d611abdb0090277a778c%2C00.html&admit=716493758+1064843859426+28353475
############################################
#
############################################
Finally, you may be losing a O/S disk, /dev/dsk/c1t6d0 or /dev/dsk/c2t6d0 or a SCSI controller, c2 or c1. Refer to LOGTOOL, ioscan (* NO_HW *) and syslog.log. Also '...dd...' the disks:

dd if=/dev/dsk/c1t6d0 of=/dev/null count=1000000

Attach the results from logtool:

STM > TOOLS > UTILITY > RUN > LOGTOOL > FILE > VIEW > RAW SUMMARY

[Ralph Grothe]

Sorry friends,
I have well noticed that two responses to my thread were posted but I simply cannot read them.

WEBMASTERS HELP!

I can only see thread listings on a section's main page.
Nice that you have included more table columns that show you also how many replies came in, and when this happened.
But what's that worth for if I cannot read the replies because all I can see are reply and back buttons and the like! :.-{

[Ralph Grothe]

Ok, I'm back with you.
Grr, I hate being forced to use Windoze Internet Exploder.
That's the User Agent I unwillingly chose right now.
It's not funny that web developers drop support for ole Mozilla in favour of IE.
Come on this is a Unix Website.
Why on earth should we be forced to use M$ clients?

Now that I can read your replies I can check the recipes and respond.

Roland,

I changed my globbing pattern as you suggested, but I didn't miss out presumably differently named vgs with my first vg* globbin.
The result (i.e. No. of devices) is the same, but good hint anyway.

Btw another annoying result of the usage of IE is that I have to clumsely cut and paste to Windoze's clipboard from Exceed's.
Many more mouse clicks than before.
(I don't like mice either)

Michael,

I "chmod o+r /dev/vg00/group", but vgdisplay keeps complaining.
Anyway I issued it as root before.

All my disks seem claimed.

# ioscan -FCdisk|grep NO_HW
# ioscan -kFCdisk|grep -v CLAIMED
#

The reading of 100 MB from the raw devs don't give evidence of broken disks either.

# vgcfgrestore -f /etc/lvmconf/vg00.conf -l
Volume Group Configuration information in "/etc/lvmconf/vg00.conf"
VG Name /dev/vg00
---- Physical volumes : 2 ----
/dev/rdsk/c1t6d0 (Bootable)
/dev/rdsk/c2t6d0 (Bootable)
# dd if=/dev/rdsk/c1t6d0 of=/dev/null bs=1024k count=100
100+0 records in
100+0 records out
# dd if=/dev/rdsk/c2t6d0 of=/dev/null bs=1024k count=100
100+0 records in
100+0 records out

[Ralph Grothe]

Roland,

how did you manage to get lines of whitespace in your reply?

It looks as if the Webserver's CGI, servlet or whatever is removing any white lines from posts before inserting them in the database's tables.

Is client supplied HTML markup supported by now?
I mean would this

give me a new paragraph?

[Ralph Grothe]

Tom,

besides many warnings (none of which relate to the disks in vg00) I get these inabilities to stat disks.

# vgscan -p 2>&1 >/dev/null|grep stat
Couldn't stat physical volume "/dev/dsk/c0t1d0":
Couldn't stat physical volume "/dev/dsk/c8t10d0":
Couldn't stat physical volume "/dev/dsk/c8t11d0":
Couldn't stat physical volume "/dev/dsk/c8t12d0":
Couldn't stat physical volume "/dev/dsk/c8t13d0":
Couldn't stat physical volume "/dev/dsk/c8t14d0":
Couldn't stat physical volume "/dev/dsk/c8t15d0":