Changing Domain ID

[Darren Gibbs]

I need to change the Domain ID on one of my Brocade 12000's and I'd like to do it with no downtime. This will change the hardware path on exactly half of the LUN's on the system. I have redundant paths to an XP1024 in which I alternate primary paths. Here's my plan:

1. Vgreduce all LUN's accessed via switch B
2. Remove all special files associated with those LUN's
3. Change the Domain ID on the switch
4. Run and ioscan and insf -e on every host to identify those LUN's
5. Identify the new card instance id of the LUN's and update vgextend scripts
6. Run several iterations of vgextends and vgreduces of primary and alternate paths in order to get them in the right order.

Does anyone think that the applications, mainly Oracle and Sybase, will have any issues with the PV link changes? I figure LVM will switch back and forth about 6 times. I must ensure there will be no downtime for this activity.

Also, what is the best way to clean up those old device paths so that demsg stops reporting errors?

[James R. Ferguson]

Hi Darren:

This sounds OK. You shouldn't have troubles with ORacle or Sybase as long as the one remaining pv_link remains operative. I'd choose as quiet a time (I/O wise) as possible.

I'd use 'rmsf' to remove the device files you don't want.

I wouldn't worry about 'dmesg' nor 'syslog' reports of pv_link switches since your are expecting them. Remember that 'dmesg' is a finite, circular buffer which will only be overwritten as necessary. Hence there may be a "final" pv_link switch message in it when you are done.

Regards!

...JRF...

[Eugeny Brychkov]

Darrn,
I would use 'ioscan -fn' instead os simply 'ioscan' to force system rescanning hardware;
Cleanup NO_HWs - reboot only, or, if you wish, monconfig reconfiguration (reboot better).
Applications should not notice pvlink change as soon as they're accessing volume group (NOT raw volumes) device files and all these switching operations are behind LVM and VG.
Although I do not recommend to make these changes on the fly - shutdown applications, do backup and apply changes
Eugeny

[Darren Gibbs]

Actually, the Sybase databases all use raw devices. We had an issue once when a Brocade 2800 was inadvertently rebooted and a few of the Sybase ASE crashed. We attributed this to the fact that the Brocade's were configured in an Arbitrated Loop configuration. Once we migrated those to Fabric PT-PT we couldn't recreate the problem.

[Michael Steele_2]

1) Note this command:

pvchange -s n/y /dev/dsk/cXtYdZ

For its possible that switch B could have either designation, primary or alternate, and this will allow you to switch pvlinks from either designation.

2) Note this command:

lssf /dev/dsk/c
echo rmsf -a /dev/dsk/cXtYdZ

Make a script and map out all of your commands first and remove 'echo' when ready.

Note -a option may be needed or the devices 'may' be remade on reboot.

5) I'd put all of this into scripts using the 'echo' command ahead of time. It'll speed up the final execution time dramatically. You can even parse out an ioscan for this identification purpose.

One final note:

fcmsutil /dev/td0,1.

Note the I/O reads and writes. Are they disproportionate over one FC / HBA?

Balance the I/O activatity over the FC / HBA with a round robin PVlink algorithm: primary alternate alternate primary. So maybe add in the new PVlinks and make them primaries with the pvchange command:

pvchange -s n/y /dev/dsk/cXtYdZ