How to prepare for device file re-numbering

[Tim Medford]

I have an L3000 box running 11i. For storage we connect to an IBM shark through a switch. The switch plugs into dual A5158A fibre cards in the HP box.

Since this is the only device on the switch we have decided to plug directly into the shark and use the switch elsewhere.

I'm concerned that when we unplug the switch and plug in directly, the hardware numbers for the LUNs and device file numbers may change. I'm not really certain what will happen honestly.

I've used LVM to setup the volume groups and logical volumes. I have defined primary and alternate paths through each fibre card when they were built.

Is there an easy way (like vgexport) to quickly rebuild everything if the device file names change? What should I do in preparation before we make the change?

Thanks in advance,
Tim

[Michael Tully · Accepted solution]

Hi Tim,

Your hardware paths will definitely change. Using switches and direct connects uses different types of hardware pathing. Generally speaking using a switch is in 'fabric' mode and direct connect 'arbitrated loop'

If you run 'vgexport' and then 'vgimport' after changing the connection, you should see all of your disks (different addresses) with all of your volume groups and logical volumes.

My suggestion is that you use the -s option when exporting. When importing using -s the vgimport program will search each of the LUN's for matching 'VGID' numbers to formulate the contents of the volume group.

Make sure that you have at least one good backup.

# ioscan -fn >/tmp/wrk1 (get a snapshot of the existing hardware)
# vgexport -m vg01.mapfile -v -s /dev/vg01 (one for each volume group)

change your connections.
Run ioscan to get the devices.
Check that you see the same number of LUN's as before.
Using the mapfiles as saved from before. This has to be done for each volume group.
# mkdir /dev/vg01
# mkdir /dev/vg01/group c 64 0x010000 (try to match the numbers used before)
# vgimport -m vg01.mapfile -v -p -s /dev/vg01 (preview mode first)
if no errors
# vgimport -m vg01.mapfile -v -s /dev/vg01
For any filesystem that has a mount point you may have to run a 'fsck' before mounting.
# vgchange -a y /dev/vg01

# mount -a

HTH
Michael

[Michael Steele_2]

I agree with the above.

[Tim Medford]

Thanks for the replies. Just one follow-up question.

If I get new hardware paths, I'm assuming the physical device file names will change too right? I mean the /dev/dsk/cXtXdX numbers.

It seems I also need to use the -f option on the vgexport so that I can change these numbers. I would then use the -f with the new infile afterwards. Otherwise I don't see how I can tell vgimport about the new device files?

Does this make sense?

Thanks

[Steven E. Protter]

Yes the entire point of the renumbering and the outcome is to change the device files.

I've done this procedure when a tape drive showed up as /dev/rmt/2m due to a tape drive being in the system previously on a fiber cable.

The point of the procedure was for the local drive to be default /dev/rmt/0m

Same thing for disks if you use the procedure for that purpose.

Micheal's post is so good, I'm bookmarking it.

SEP

[Michael Tully]

Hi Tim,

There is no need to use the -f option at all.

As long as you use the -s option is used, the vgimport (again using -s) will search for matching 'VGID' numbers pertaining to the volume groups. The device numbers will change, but I am quite sure it will only be the 'vbus' (virtual bus) number being 'cX'. The 'tYdZ'part of the device should remain the same.

Regards
Michael

[Tim Medford]

Thanks again. I think I will use the -f in addition to -s just in case. It shouldn't hurt anything to capture the device file names, and then I can always use it on the vgimport if I need to.

And I agree, Michael's post is great! It was exactly what I need. I think it should be a mknod command instead of mkdir on the group file though.

Tim

[Tim Medford]

Just wanted to follow-up on this thread. I completed the switch last night and everything worked fine. I added a couple extra steps so I thought I'd just throw them in here for reference:

Shutdown to single-user mode.

Deactivate volume groups:

-vgchange â a n /dev/vg01
-vgchange â a n /dev/vg02

Export existing volume groups:

-vgexport â m vg01.mapfile â v â s /dev/vg01
-vgexport â m vg02.mapfile â v â s /dev/vg02

Disable fibre cards:

-fcmsutil /dev/td0 â f disable
-fcmsutil /dev/td1 â f disable

Switch cables

Enable fibre cards:

-fcmsutil /dev/td0 enable
-fcmsutil /dev/td1 enable

- ioscan â fnC disk â Verify correct number of LUNs

Create new device files:
- insf -e

Make directories:

- mkdir /dev/vg01
- mkdir /dev/vg02
- mknod /dev/vg01/group c 64 0x020000
- mknod /dev/vg02/group c 64 0x030000

Import volume groups:

vgimport â m vg01.mapfile â v â s /dev/vg01
vgimport â m vg02.mapfile â v â s /dev/vg02

Activate volume groups.

-vgchange â a y /dev/vg01
-vgchange â a y /dev/vg02

I rebooted afterwards and did encounter 1 error. The boot process stopped and said /etc/ioconfig was inconsistent with /stand/ioconfig. It then rebooted itself and eveything came up fnie the second time.

I should have sync'd up the ioconfig files before booting

[Geoff Wild]

I would also add to your plan to remove old device files with the rmsf command.

Thanks for posting your results.

Rgds...Geoff

[Lacrosse]

Tim I am getting to do a very similar task but in reverse I will be dropping a brocade switch in between an existing direct connect VA7410 I was wondering if I could ask you a few questions about the setup/proceedures etc.... if that would be ok would it be possible for me to call you as it would be easier to explain what I am trying to do....Thanks