Effect on HP-UX SAN Attached systems during FC Switch Port Relocation

[Alzhy]

I have yet to confirm whether our vendor actually suggested that our HP-UX servers stay up when our SAN folks "re-arranged" our Server connections to our switches But it does appear it resulted in a number of problems.

Our SAN team assured us the servers can stay up during their SAN port reorganization but the servers needed to be "quite" -- which we complied with. However, after the reorg.. all of our servers started having all sorts of problems.. Both LVM and VxVM managed storage sets (filesystems) are no longer accessible despite reboots.. Most were easily addresed rectified but some of our FlashSnap/host based mirror configs will need to be resynch'ed fully.

So my question is -- was our SAN team's suggestion valid? Should we have been more prudent and shut down the systems while they did their cabling/port changes. And on server startup.. is there anything FCMSUTIL wise that we need to do?

[Pete Randall]

Nelson,

The answer seems pretty obvious to me!

Did the san team's rearrangement cause device file address changes? Is that what caused your inaccessabilities? I would expect that it wouldn't matter whether the servers were up or down - the addresses are going to have to be re-configured after the change and the servers will end up needing to be re-booted anyway, so you might just as well have them down.


Pete

[Alzhy]

That's what I thought as well.
Thanks Pete!

[Simon Hargrave]

When switch ports get moved around, the hardware path of the device changes, based on port number etc. If you change these, your devices get different hardware paths, therefore get redetected with different instance numbers, and the device names change.

The safest way is to unmount all SAN volumes, vgexport them with maps, then vgimport them with the changed disk id's.

I guess if you wanted to do it all "online" then you could proceed somehow like (assuming you have dual-pathed everything):

FOREACH switch port
vgreduce volume groups with any paths down this port
swap fibre into new port
ioscan
insf
vgextend volume groups with the "new" paths
NEXT switch port

Could very easily cause headaches though, so I'd recommend the shutdown and export/re-import route.

[TwoProc]

It seems that the SAN is really very picky with device files, especially during moves/changes, which result in what you're describing. And, it seems to happen half of the time, instead being the rare exception. So, I just have our team do a backup beforehand and schedule a full shutdown - just to be conservative about the whole thing.

[Alzhy]

Thanks!

I did notice the VxVM machines fared well since the configs are not dependent on the cXtYdZ values. We literally did not have to do anything on them .. except for our mirroed environments. All of LVM configs failed and even if recovery was easy.. it took quite a bit of time concidering the number of servers hooked up to our SAN.

[Robert Bennett_3]

We are finally done with our upgrade of the SAN.

We moved our environment from a single director, single fabric to a dual director, dual fabric - all hot.

Issues - as mentioned here previously- hardware address changes.

Work Flow-
- move ports
- ioscan -fn
- insf -e
everything worked well - but an ioscan still showed NO-HW on the old addresses. This weekend we had the "opportunity" to reboot all of our SAN attached servers (~25) and cleared up all the NO_HW's in the ioscans.

Maybe we were lucky - but it worked for us.

[Richard Pugh_3]

Not sure what SAN switches you are using but if these were Cisco MDS switches, you can persistently bind the HP host ports.

This is not the SAN being picky, it is the HPUX server. A SAN issues each port an FCID (Fibre Channel ID) to identify each device on the switch for routing purposes. With the Tacyon controllers, once a device is logged off of a switch and then back on, it queries the switch for a new ID.

[Suraj Singh_1]

Hi,

SAN attached LUNs have the device files which depends on Fabric Port. As soon as the port changes, the device files of the LUns are bound to change. Though VxVM doesn't depend opeon the device file (as it maintains its own disk id on each of the disks which are brought into its control), but LVM depends on these device names.

The best option in this case could have been to unmount all SAN volumes, vgexport them with maps, then vgimport them with the changed disk id's.

Regards,

[Victor BERRIDGE]

Hi Nelson,
Each time someone changed a brocade switch or port (once it was the bay while staying on same switch..) I went through a process of export/import vg...
I usually make map files before any of these manipulations in case...
Once (lately) though after moving to different location all was fine...
My point of vue is that at a certain point your systems even without activities would have complained because it is not seing its PV anymore...
I remember having no trouble but I did disable all my SAN vg before the SAN intervention.

All the best
Victor