Sequential Steps require after creation of vdisk at EVA

[S.S.]

Hi Experts,

Can someone help me the exact steps to do after creating the vdisk LUN on EVA8000?

1)how to scan for new device files.
2)how to create the vg, lv and fs
3)what are the changes to be done on multipath.

We are using the device mapper multipath.

Please help me with exact steps.

Thank you!

[Matti_Kurkela · Accepted solution]

> 1)how to scan for new device files.

For steps, please see chapter 9 of this RedHat document:

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Online_Storage_Reconfiguration_Guide/index.html

> 2)how to create the vg, lv and fs
> 3)what are the changes to be done on multipath.

You should FIRST set up multipathing, THEN set up LVM VGs and/or filesystems on multipath devices.

HP provides a "device mapper multipath enablement kit" for EVA storage. Download an appropriate version for your Linux distribution from here:

http://www.hp.com/go/devicemapper/

Generic information on device-mapper multipathing (you'll need this to understand the other documents):

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/DM_Multipath/index.html

HP EVA-specific information, including the installation steps for the Enablement Kit:
http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c01762511/c01762511.pdf

Once you have multipathing set up, each multipath device should work just like any other disk device (although a very reliable one). So the VG/LV/FS creation steps are just the same as on normal SCSI disks.

Please see Chapter 5 ("LVM Configuration Examples") of this document:

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Logical_Volume_Manager_Administration/index.html

MK

[Alzhy]

Here it is.

Make sure you have installed HP Device Multipath Enablement Kit ( Google and Download/Install "HP Device Mapper Multipath"). Follow Instructions parameters recommended for your flavour of F-HBA. Last time - you can download from here: http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=3559652&prodTypeId=12169&prodSeriesId=3559651&swLang=13&taskId=135&swEnvOID=2026

Your /etc/multipath.conf should have a block that contains:

# For EVA A/A arrays

device {
vendor "HP|COMPAQ"
product "HSV1[01]1 \(C\)COMPAQ|HSV2[01]0|HSV300|HSV4[05]
0"
path_grouping_policy group_by_prio
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
path_checker tur
path_selector "round-robin 0"
prio_callout "/sbin/mpath_prio_alua /dev/%n"
rr_weight uniform
failback immediate
hardware_handler "0"
no_path_retry 18
rr_min_io 100
}


Make sure multipathd is runing:
#service multipathd status


And Everytime you present Vdisk to your Linux Machine (RHEL I presume), you need to do the following:

1.) Add Entry in /etc/multipath.conf in the multipaths{} section something like:

multipath {
wwid 3600508b40006a4640000a000011b0000
alias EVA7_Disk001
}


2.) Scan your SAN for the new LUNS/Vdisks:

# for hba in `ls -d /sys/class/fc_host/host*`;do
FCHOST=`basename $hba`

echo "Scanning $FCHOST"
echo "- - -" > /sys/class/scsi_host/${FCHOST}/scan
done

3.) multipath -v2

4.) multipath -ll

5.) You should now have your friendly named VDISK as a device /dev/mapper/EVA7_Disk001

ANd you can now do whatever with it under LVM.

[S.S.]

Dear Matti Kurkela and Alzhy,

Thank you very much with your valuable inputs.

Let me explain more about my scenario:
We are planning to do the migration for our storage EVA ie., from EVA8000 to EVA8400.
Totally, 7 servers are accessing EVA8000 in which 5 are HP-UX servers and 2 are Linux Servers.
I know the proper steps for Presenting the LUNs for HP-UX servers but don't have idea for Linux Servers to do.

So, we are planning to switch our EVA8000 to EVA84000 in which the Linux servers should see the new device files.

I have done some enhanced search in google and found 2 nice articles. I will share those when i go to office. Now i am posting the mssg from home.

As per the articles, following are the steps have to be done when the linux box can see with new EVA8400.

[1] In new EVA, after presenting the LUNs to linux box then scan for new devices files. Yesterday, i have installed evainfo tool on Linux server to easily find out the physical disks corresponding to WWN number of vdisk.

[2] Just run the vgscan command which will automatically links with existing VGs.

[3] Have to update the multipathd.conf file with the newly generated WWN number.

May i know, the above steps are correct? If not please correct me with exact steps.

Yesterday, in our existing setup Linux seeing EVA8000.
I have found that:
1) when i ran vgscan, pvscan and lvscan it just showing the root VG and root VG pvs but unable to see the other VGs.
But when i executed the 'dmsetup ls' command i can see the rest of VGs also and from output of 'df -h' can still access the corresponding filesystem of other VGs.

2) I was unable to see the output for multipath also, with 'multipath -ll'
showing no outputs.

May i know what could be the reason?

Please help me for these two issue which i summarize you:

1) During migration time how can we do vgexport and vgimport of VGs.
2) Unable to see the output for multipath and vgscan showing only root vg.

When i checked in other Linux server, i can see the successfully output of vgscan,pvscan and lvscan.

Our RedHat OS is RHEL4 update 4.

I would be very thankful if you can give me clarity steps for my problem.

Waiting for your quick response.

[S.S.]

Dear Matti Kurkela and Alzhy,

Yesterday, just for testing i have created one 10gb vdisk in EVA8000 and presented to the host.
Created VG, LV and FS.
I have successfully created and also exported and removed.
May i know how can i remove the device files generated for physical disk?

Special thanks to you in advance.

[Matti_Kurkela]

>[3] Have to update the multipathd.conf file with the newly generated WWN number.

This is required only if you have specified custom names for your multipath devices.

If you don't specify custom names, the dm-multipath subsystem will auto-generate names using one of the two possible formats:

- if friendly_names is enabled in multipath.conf, each time the system sees a new disk, it will record its WWN to /var/lib/multipath/bindings and assign it a name like /dev/mapper/mpathN, where N is a number. This name will be forever associated with that WWN, unless the /var/lib/multipath/bindings file is deleted.

- If friendly_names is disabled, the multipath devices will be named /dev/mapper/. If the storage system is configured correctly for Linux, this number is unique for each LUN and storage system, and will stay the same for the entire lifetime of the LUN.

> 1) when i ran vgscan, pvscan and lvscan it just showing the root VG and root VG pvs but unable to see the other VGs.

Check the filter rules in /etc/lvm/lvm.conf. If the filter is set to allow specific devices only, you may need to adjust the filter each time you add more disks/LUNs.

> 2) I was unable to see the output for multipath also, with 'multipath -ll'
showing no outputs.

Is the multipathd daemon running?
Run "multipath -v2": it will force the multipath subsystem to search for disks again. If it produces any output, perhaps the multipathd daemon was not running (or was using an old configuration, if it was not restarted after editing the multipath.conf file).

> 1) During migration time how can we do vgexport and vgimport of VGs.

I see you're maybe thinking about HP-UX LVM procedures.

In Linux, the LVM subsystem is similar to HP-UX LVM on the surface, but the implementation is very different. In Linux, there are no map files, and it is often not necessary to use vgexport/vgimport at all!

Background:
In HP-UX, the name of the VG is stored in the LVM configuration, not in the VG itself. An exported HP-UX VG is a nameless entity: it can be identified only by a VGID stored in the map file, or by listing all the PVs that belong to the VG. Exporting a VG removes the system's knowledge about the VG name, and creates a map file which is supposed to be moved along with the VG.

In Linux, the name of the VG is stored in the VG metadata in the beginning of each PV. This allows Linux to automatically use a correct name for a VG even if that VG has just been moved from another system. But it also creates the possibility of VG name conflicts: if you move a Linux VG from one system to another, there may already be another VG with the same name on the destination system.

In Linux, vgexport/vgimport is *only* a mechanism for avoiding the VG name conflict. When a Linux vgexport command is run, it does *not* remove the system's knowledge of the VG: it only puts a special flag to the VG metadata. The flag has the following meaning: "This VG must not be activated until this flag is removed by the sysadmin. If there is a VG name conflict, use the other VG with the same name, not this one."

Linux LVM has a specific command for renaming VGs: "vgrename".

> 2) Unable to see the output for multipath and vgscan showing only root vg.

See above.

> May i know how can i remove the device files generated for physical disk?

To tell the system that e.g. /dev/sdx is gone and won't be coming back, use this command:

echo 1 > /sys/block/sdx/device/delete

MK

[S.S.]

Hi Matti Kurkela,

Thank you for your reply.

I have noticed that the luns in linux server is addressed by dev-mapper-multipath. Due to because of that i was unable to display the vg for those.

So, in that case, may i know what are the steps should i have to do during san migration ie., switching the linux server to new eva?

Please advise me and reply me as early as.

[S.S.]

Hi Matti Kurkela,

The filesystems are mounted with the following:

/dev/mapper/dwarch-lvdwarch
50G 19G 29G 40% /dwarch
/dev/mapper/dwfbackup-lvdwfbackup
493G 162G 306G 35% /dwbackup
/dev/mapper/dwexedata-lvdwexedata
788G 567G 181G 76% /oradwexe
/dev/mapper/dwredo-lvdwredo
9.9G 684M 8.7G 8% /dwredo

My vendor said that these are addressing by device-mapper-multipath and told that due to that i was unable to do vgdisplay.

Is it possible to create filesystems without lvm?
Sorry to disturb you with number of questions.

[Alzhy]

SS,

Your friends in your migrations and easily identifying your LVM objects are:

pvscan
pvs
vgscan
vgs
blkid

Do a man page on those and you will be enlightened.

You will also have to do some reaing on /etc/lvm.conf

And yes, you can create ext2/ext3/ext4,etc filesystem on top of *any* disk device in Linux outside of LVM.

And once again, please be familair with multipath.conf and the usage of ffiendly names or identifying the multipath names and their subpaths.

You may also want to install evainfo tool on your Linux machines.

[S.S.]

Hi All,

I would like to post the steps done during our SAN migration which will help others.

Actually, we are migrating our current EVA to new EVA and HP used CA to replicate the data from existing EVA to new EVA and break the CA.
And our aim to switch our existing Linux Servers from current EVA to new EVA and following are the steps done.

We have 2 Linux servers accessing the data in EVA. 1 Linux server is without LVM for filesystem and 1 Linux Server is with LVM for filesystem.

Steps done on Linux Server without LVM:

1. umount all SAN filesystems.
2. power-off the servers.
3. Done Zoning and switch the fc cables from existing EVA to new EVA.
4. Power-on the server.
5. Checked the SAN disks using lssd command and can see the new EVA disks.
6. ran only multipath command to create new mappings.
7. then executed the multipath -ll command to see the newly created devices.
8. Edited according on /etc/multipath.conf file.
9. Restarted the multipath services.
10.executed the mount -a command to mount the filesystem.
11.check the filesystems using df -h command and those filessytem are accessing the data from new EVA.

Steps done on Linux Server with LVM:

1. umount the SAN filesystem.
2. power-off the server
3. done the zoneing and switch the fc cables from existing EVA to new EVA.
4. power-on the server.
5. Check the new SAN disks using lssd command.
6. As it is used the LVM for filesystem creation, it will automatically uses the new SAN disks and mounted automatically.
7. Check the filesystem using df -h and it is already mounted.