Problem with vgcreate -- a touch of fear !!

[James Hager]

Greetings, and thanks in advance:

I am trying to create a new volume group on my RP7400. I have a CLARiiON CX400 that is attached to this, and I am essentially setting up a new LUN. The LUN is created, and I have found my new "disk" using ioscan, created the physical volume, and made the corresponding /dev directory. The volume/disk I am using even shows up in the nodes when I do an ll /dev/emc*/group command. Mknod had no problem that I could see. The problem comes in when I try to run the vgcreate command as follows: vgcreate /dev/emc08 /dev/dsk/c5t0d7. I get the error "vgcreate: cannot open the control file /dev/emc08/group: no such device".

The last time I saw this was in the test server, which was indicating that I had reached the maxvgs limit. According to what I can tell on our live server, this isn't the case. The disk also has no entry in the /etc/lvmtab that I can see (which, again, concerns me).

Last thing of interest: when I look at the groups (ll /dev/emc*/group), I see hex values of 0x020000 - 0x0a0000 (0x0a0000 is the one for the vg I'm trying to add), with the exception that 0x030000 is not in the list.

How do I get around this problem? I would prefer not to have to recompile the kernel, as I don't have the time to really test doing that . Any other suggestions?

Thanks again--

--James

[Geoff Wild]

Uh....there should be only 1 group file with a single minor number.

I would try again..

vgexport /dev/emc08
mkdir /dev/emc08
mknod /dev/emc08/group c 64 0x0a0000

(make sure 0x0a0000 doesn't already exist already: ll /dev/*/group )

BTW - for naming standards - I suggest /dev/vgemc08

Rgds...Geoff

[Mel Burslan]

instead of running ll /dev/emc*/group, if you run ll /dev/emc08/group command, what do you see ? Is it actually a group file ? permissions and ownership correct ?

Worse comes to the worst, delete the group file under emc08 directory and recreate it. That would be the first thing i'd do as it sounds like the LVM doesn't like this entry.

[Steven E. Protter]

proper procedure to create a volume group.

cd /dev
mkdir vgname

# example mkdir vg03

mknod vg03/0x0300000

Now you can create a volume group with vgcreate.

If there are extraneous grup files out there you may need to get rid of them, not not before backing up your lvm structure with vgcfgbackup

I'm thinking you didn't have a cookbook and all you need is the mknod command to proceed.

SEP

[Raj D.]

Hi James ,

Check for the group file.
ls -l /dev/emc08/group (It should be of unique id )

And possibly you need to recreate with mknod and go from the scratch,

Cheers,
Raj.

[Alan Meyer_4]

You mention that you have not exceeded MAXVGS, but if the MAXVGS value is still set to the default of 10, then 0x0a0000, which is 10 in hex is too high. The max a minor number can be MAXVGS - 1. (Remember the 1st available minor number is 0x000000).

So, what is the value of MAXVGS???

[James Hager]

Geoff:

I was a bit unclear on the group/minor numbers. There is only one number per entry, i.e., emc08 is only assigned 0x0a0000. My bad.

I can't vgexport, since the entry doesn't exist in the lvmtab. Trying to do mkdir or mknod returns errors that both already exist.

As far as the standard goes, I would agree with you...but I inherited this system, and want to keep with the existing structure.

*shrug*...am I missing something?

------------------------------

Mel:

Running a ll /dev/emc08/group returns what looks like a proper group file with correct permissions, at least as matched against the other entries. Only difference I see is that the other entries (such as emc07) have rv01 and v01 entries under the dirs, while emc08 doesn't.

Thanks, guys....I'm still trying to work this out.

[Geoff Wild]

Then - just remove the directory - then start over with steps above...

rm -rf /dev/emc08

Rgds...Geoff

[James Hager]

Alan:

Good point, I did notice that. Is there a way to find out if 0x030000 is assigned to something? If not, can I delete my assigning of emc08 to 0x0a0000 and assign it to 0x030000? All of the other entries in the /dev/emc*/group have hex numbers; in the series of 0x020000 to 0x0a0000, only 0x030000 is missing. If I can use that one, safely and with some assurance that I'm not bombing something else in the system, I'll use it.

My maxvgs parameter is still set at the default of 10.

Thanks!

[Alan Meyer_4]

I would say you can use 0x03, but 0x00 & 0x01 also seem available too.

[Mel Burslan]

James,

As Alan said, whichever minor numbers you are missing in the sequence, can be used for a new volume group. They do not increase linearly as you add new groups. You can fill the holes in the lower numbers vacated by exporting or removing old vg's.

[Geoff Wild]

minor number is actually in HEX.

Therefore, when you create a vg with say a name of vg79, it's minor number with our current standard would be 0x790000

What's wrong with that you say? well, what if the kernel parameter MAXVGS is set to 80?

79 HEX is 121 decimal - which is 42 vg's more then you can have with MAXVGS at 80 (starts with vg00).

So, the minor number for vg79 should really be: 0x4f0000

So, MAXVGS=10 - means 00 to 09 - not 0a

Rgds...Geoff

[Alan Meyer_4]

Mel, I appreciate you, you always seem to keep me in line ~smile~

[James Hager]

Thanks, everyone:

Ok, I deleted the emc08 as directed, then did the pvcreate, made a new directory, and did a mknod using 0x03. Finally, I did the vgcreate(vgcreate /dev/emc08 /dev/dsk/c5t0d7), which worked nicely. Once I did the lvcreate, it told me I needed to do an lvextend (which I expected), but looks like that worked also. However, now I'm choking on the newfs command. I run "newfs -F vxfs /dev/emc08/lvol1", and I am told that it is not a character device. Sure enough, if I go to /dev/emc08, lvol1 has permissions of brw-r-----. I have gone through this on the test server, and didn't have this problem (granted, it was a year ago, but...).

I admit, I am more of a newb at this than I want to admit, but still....I'm working on getting my head around this, so that I don't make the same mistake again later.

Thanks, and I'm now going to go back and start handing out points to everyone! (Oh, and the good news is that this *ought* to be my last hurdle....after this, I just need to create the mount point on the server and mount the new dir up, I think...)

--James

[Mel Burslan]

use raw volume for newfs command like:

/dev/emc08/rlvol1

[Torsten.]

James,

it is always helpful to take a look at the man pages.

man newfs or man newfs_vxfs

it is telling you the options.

(use *r*lvol -> /dev/vg/rlvol/..)

It may be also a good idea to set the large file option, so you are able to create files >2GB.

newfs -F vxfs -o largefiles /dev/emc08/rlvol1

[Mark Greene_1]

James,

You didn't say if you have powerpath; I'm going to assume that you do.

Even though the /dev/rdisk/ file is created by powerpath, you still have to do the pvcreate in HP-UX in order for the correct LVM info to be written, which is what vgcreate is looking for and couldn't find. I see later in the thread that you have since done this. You also didn't state if you ran insf -e -C disk after the ioscan. If you haven't done so, you will need to.

You are also going to want to do a pvchange on the disk using the "-t 180" option to set the default timeout to 3 minutes in order to alieveate false timeout detections by PowerPath.

Finally, don't forget to run powermt check, powermt config, and powermt save to complete the setup on the PowerPath side.

Good luck,
mark

[James Hager]

Greetings and thanks to all who helped out with this! I finally got it resolved, and it turned out to be a combination of a couple of solutions: I used the 0x03 value, and that worked out well. The other issue that I had going on was that I was trying to assign too much space to the lv when I was setting it up. Once I fixed that, all was well. I am happy to say that I am now the "parent" of a happy new volume, and our crisis has been averted (and I have learned a LOT).

Thanks again to all for the great answers!

(And, btw, I *did* read the man pages. Doesn't mean that I fully understood them, though....a lot more NOW, at any rate)

Thanks!!

--James