ioinit & VxVM devices

[borut kurnik_1]

Hello!

I've got two rx4640 with identical HW, using 4 paths to see LUNs on EVA.

I used ioinit and now my HW addresses, Instance
numbers and devices files are synchronized, so
'ioscan -funC disk' gives me the same results on both machines, but with VxVM:

#vxdisk list # first
DEVICE TYPE DISK GROUP STATUS
c5t0d1 auto:none - - online invalid
c5t0d2 auto:none - - online invalid
...

# vxdisk list # second
DEVICE TYPE DISK GROUP STATUS
c9t0d1 auto:none - - online invalid
c9t0d2 auto:none - - online invalid

# vxdisk path # first
SUBPATH DANAME DMNAME GROUP STATE
...
c5t0d1 c5t0d1 - - ENABLED
c7t0d1 c5t0d1 - - ENABLED
c9t0d1 c5t0d1 - - ENABLED
c11t0d1 c5t0d1 - - ENABLED
c5t0d2 c5t0d2 - - ENABLED
c7t0d2 c5t0d2 - - ENABLED
c9t0d2 c5t0d2 - - ENABLED
c11t0d2 c5t0d2 - - ENABLED

# vxdisk path # second
SUBPATH DANAME DMNAME GROUP STATE
...
c9t0d1 c9t0d1 - - ENABLED
c5t0d1 c9t0d1 - - ENABLED
c7t0d1 c9t0d1 - - ENABLED
c11t0d1 c9t0d1 - - ENABLED
c9t0d2 c9t0d2 - - ENABLED
c5t0d2 c9t0d2 - - ENABLED
c7t0d2 c9t0d2 - - ENABLED
c11t0d2 c9t0d2 - - ENABLED

How do I fix that?

Thanks,

Borut

[Zinky]

Borut,

I actually do not see what the issue is.

You're using an EVA which is friendly to industry standard multi-pathing solutions like VxVM DMP right? Is this an EVA4/6/8K or an upgraded EVA 5K/3K?

With VxVM, once your diskgroups (vxvm lingo for volume group) are carved, ANY changes to cXtYdZ matters not as long as VxVM sees the members of the group. The "ordering" of which path is treated as "primary" in your listing from vxdisk list again matters not as VxVM DMP is "intelligent" enough to decide which is the primary due to its awareness of the innards of the array.

Let me know if you still have issues.

[Albert_31]

Hello Borut,

What you are seeing are the VXVM objects which are in the VXVM database..generated against the iodevices.. if there is no data on the disks mentioned above, then you can remove them and initialize them again to reflect the new device files... else you can do the following

# vxdg deport
# vxdctl enable
# vxdisk list
# vxdg import

This should help..else let me know...

regards

albert

[borut kurnik_1]

Hello!

Sorry, guys, no issue here.

Just wanted to know why VxVM chooses different
primary paths with two machines with completely
identical HW configuration ...

Thanks,

Borut

[Albert_31]

when did you do the ioinit procedure?

[borut kurnik_1]

Hello!

Well, Albert. I wanted to achieve device
euity between the two machines when creating
LVM group for lock disk. Just want some
discipline (perhaps sometimes in the
wrong places :-) ).

I also checked the disks with VxVM's tools
('vxdisk list') and I wanted to have the
same view there also, so I did the ioinit on
node 2 and synchronized disk instance numbers to HW addresses, then recreated device files
(insf)...

VxVM still shows the same ...

But what is interesting is that when I did
some tests to a CFS disk (without dmp), both nodes used the same path - c9t0d2
(vxdmpadm iostat).

Of course, don't want to waste anybody's time, I was just curious how VxVM
picks the primary path...

I know, I'll have some more reading to do
(RTFineM).

Regards,

Borut

[Mridul Shrivastava]

I have given the procedure for the same in the below mentioned thread:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1027043

Have a look.

[Mridul Shrivastava · Accepted solution]

The ioconfig command provides the mapping between instance numbers used
by the kernel and the information the I/O system uses to communicate with
peripheral devices (hardware paths). Two copies are maintained
(/stand/ioconfig and /etc/ioconfig).

At boot time the ioconfig information is stored in the io_tree kernel data
structure (see ioinit(1M)). The only purpose of the ioconfig is to maintain
configuration information when the system is not running. Even if hardware is
removed from the system all mappings keep in place. This guarantees that no
new device file names will appear after such changes. If removed hardware is
added back to the system the original mapping can be reused, since it is still
present in the ioconfig files.

Usually we want to change mappings for disk and lan devices. For lan devices
we change directly the corresponding lan instance numbers. For disk devices we
need to take care of the ext_bus instance numbers. The numbers of
such 'External Busses' (aka card instances) are responsible for the 'c'
numbers being part of disk device names.

Look at the following extract of an ioscan -fn:

ext_bus 3 2/0/1 c720 CLAIMED INTERFACE Built-in SCSI
target 0 2/0/1.3 tgt CLAIMED DEVICE
target 1 2/0/1.5 tgt CLAIMED DEVICE
disk 4 2/0/1.5.0 sdisk CLAIMED DEVICE SEAGATE
ST15150N
/dev/dsk/c3t5d0 /dev/rdsk/c3t5d0

The ext_bus instance number (3) is responsible for the 'c3' in
'c3t5d0'. The target (t5) and the LUN (d0), which affect the rest of
the name, are not changeable by software. Instead they map directly to the
underlying hardware configuration.


WARNING: Remapping instances is inherently dangerous and may result in an unbootable system, if not performed with care. Thus it is advisable to take a backup (e.g. using make_tape_recovery) of the system before attempting any of these procedures. It is also advisable to save the current
/etc/ioconfig file, e.g to
/etc/ioconfig.bak. Usually it is possible
to recover from failures by copying that file back to /etc/ioconfig and
/stand/ioconfig before rebooting.


This procedure requires one reboot and works without additional tools.

1. Extract a configuration template from the current ioscan output.
Execute the following command:

# ioscan -f | grep -e INTERFACE -e DEVICE | \
grep -v target | \
awk '{print $3, $1, $2}' > /infile

2. Edit /infile and change the ext_bus and lan instances as desired.
No class is allowed to get more than one line for the same instance!

3. Bring down the system gracefully to run level 1.

# init 1

4. Apply the ioconfig change:

# /sbin/ioinit -f /infile -r

The system will reboot immediately if the change is successful.
Warnings like 'Input is identical to kernel' can be ignored.

If unsuccessful, the most likely error to happen is:
"ioinit: Instance number X already exists for class XXX"

The problem is that your desired instance assignment conflicts with
an existing instance number. If that instance is bound to hardware
that is no longer visible in ioscan, then you are in trouble and
need to perform the Procedure II or III.

5. Once the system reboots, verify that all the instance numbers
were changed as expected. It may be necessary to re-import volume
groups to ensure that /etc/lvmtab contains the correct
entries. The lan configuration may need to be changed also.

Be sure to have the ioinit(1M) patch installed:
10.20: [PHCO_16407/PACHRDME/English] (or greater)
11.00: [PHCO_12555/PACHRDME/English] (or greater)

If the above mentioned procedure fails then try this one:

Reliable, requires two reboots and works without additional tools.

1. Extract a configuration template from the current ioscan output.
Execute the following command:

# ioscan -f | grep -e INTERFACE -e DEVICE | \
grep -v target | \
awk '{print $3, $1, $2}' > /infile

Make sure to store infile to the root file system!

2. Edit /infile and change the ext_bus and lan instances as desired.
No class is allowed to get more than one line for the same instance!

3. Move away the current ioconfig files and Shutdown/Reboot:

# mv /stand/ioconfig /stand/ioconfig.sav
# mv /etc/ioconfig /etc/ioconfig.sav
# shutdown -ry 0

4. Due to the missing ioconfig files the system will come to an
ioinitrc prompt. Now recreate new ioconfig files from scratch.
This prevents you from running into possible assignment conflicts.

(in ioinitrc)# /sbin/ioinit -c

5. Apply the ioconfig change with your prepared infile:

(in ioinitrc)# /sbin/ioinit -f /infile -r

The system will reboot again now if the change was successful.
Warnings like 'Input is identical to kernel' can be ignored.

6. Once the system reboots, verify that all the instance numbers
were changed as expected. It may be necessary to re-import volume
groups to ensure that /etc/lvmtab contains the correct
entries. The lan configuration may need to be changed also.

[borut kurnik_1]

Thanks, guys.