card instances

[Tom Bear]

We are gettign ready to add some new Fibre Channel cards to our systems for redundant pathing to our XP256 and tape library. We have some concerns about card instances changing and altering our present disk addresses, thereby causing us much vg rework. Can someone explain to me, or point me to a resource, how a K-class system determines card instances.

[S.K. Chan]

This might help :-

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x8f567b8d1de3d5118ff40090279cd0f9,00.html

[Sridhar Bhaskarla]

*It is always better to save the map files preferable on another server before doing any work related to disk subsystem*

Having said that, if you do not distrub your current cards/controllers/chain, you shouldn't have any problems with their instances and hence the device files.

We cannot presume the new instance numbers before installing the new cards. It is dependent on how the system scans the new hardware when it comes up. Usually for the disk controllers, it adds 2 to the current value. For ex., if you have two instance numbers 1 and 3 for the existing controllers, the new ones would be 5 and 7. But it may not happen 100%.

-Sri

[A. Clay Stephenson]

Hi Tom:

The instances numbers are determined by the hardware path. If you do an OS install and change nothing, the instances will be assigned the same each and every time. The problems occur when cards are added/moved AFTER an OS is installed. Let's suppose that you started out with two controllers in slots 1 & 4. (I'll just use the simple concept of slot 1 rather than a 10.4.XX because it's easier to follow.)
Those might be assigned instance numbers 1 and 2.(e.g. c1, c2) Now imagine that you install another SCSI controller in slot 2. It would become instance 3 (c3). The instance numbers are assigned in the order that they are 'discovered' and the search order follows the hardware path.


Now, imagine that you re-install the OS. Because the is now cards in slots 1,2, & 4 and because slot 2 is encountered before slot 4, c2 and c3 have now been swapped.

I hope I haven't confused you too much, Clay.

[James R. Ferguson]

Hi Tom:

Adding new hardware shouldn't change existing instance numbers. The issue usually surfaces later when you do a cold-installation. Instance numbers are generated in ascending order as devices are found when hardware buses are scanned.

As for LVM, it is very easy to 'vgexport' and 'vgimport' with mapfiles and a device "outfile/infile" file:

Consider this, using vg02:

# vgchange -a n /dev/vg02
# vgexport -m /tmp/vg02.mapfile -v -f /tmp/vg02.oldpaths /dev/vg02

...then...

# mkdir /dev/vg02
# mknod /dev/vg02/group c 64 0x020000
# vgimport -m /tmp/vg02.mapfile -v -f /tmp/vg02.newpaths /dev/vg02
# vgchange -a y /dev/vg02

You can capture the "oldpaths" when you export the affected volume groups, edit the device declarations to reflect the new devices, name the edited file "newpaths" and import the volume group.

Regards!

...JRF...

[Tom Bear]

Clay,

In your example, will the instance numbers change after a reboot after the 3rd controller is installed in slot2? Or would they still be discovered in the old order until a reinstall is done?

[A. Clay Stephenson · Accepted solution]

Hi Tom:

No, once assigned the instance numbers remain fixed (unless you jump through some hoops using ioscan). It's only when you do another cold OS install that you might see the instance numbers change. Also, if you reinstalled the OS using Ignite, the instance numbers would be preserved - that is one of the benefits of Ignite. In most cases, the instance numbers don't matter and in fact most people complain when their tape drive devices change. I have a hard time understanding why that is a big deal since it's so easy to change them. The biggest problems occur when you are using raw database i/o without any indirection. If Oracle is expecting /dev/rdsk/c2t5d0 and suddenly that's really /dev/rdsk/c3t5d0; that's a big deal. My convention for that is rather simple. The database expect something like /u01/oradata/file01.dbf. I then would symbolically link /dev/rdsk/c3t5d0 to /u01/file01.dbf and Oracle never know nor cares about the actual raw disk node.

Regards, Clay

[Tom Bear]

Thanks for your help everyone. I think I have enough info now to change my FC cards to the newer ones and add some additional ones without too much damage. And be able to understand, and fix, the damage I may cause!!

[Sachin Patel]

I am not agree with clay.
We have couple V-class. Once I had failed boot disk on v-class and I did cold-install all of my disk's instances was changed. I have 22 vg's and it was so much work. Luckly I had output of all vgdisplay and ioscan so once I figure out how much it has shift instance number it was easy.

Always keep paper copy of vgdisplay, ioscan etc...

Sachin Patel