topic qlogic failover & hp_rescan que? in Operating System - Linux

qlogic failover & hp_rescan que?

klemerS — Mon, 06 Mar 2006 15:17:48 GMT

Hello.

I have 2 Q?

1. on linux with 2 qlogic HBAs - with the
latest driver ( failover mode ) - if i am
looking at /proc/scsi/qla2300/0 & 1 do i need
to see the same luns on both HBAs ???( i present the luns to both HBAs ( eva5000 )
I know that in /proc/scsi/scsi i will see only
one scsi device.
2. Is it safe to run hp_rescan on production
server ?

Thanks.

Re: qlogic failover & hp_rescan que?

Steven E. Protter — Mon, 06 Mar 2006 15:45:04 GMT

Shalom klemer,

1. If the luns point to the same disk and the intent of your setup is to provide redundant access to both cards to the same disk, then yes you should see the same luns. They will have different physical paths though.

2. I would not. That doesn't mean its not safe, but I don't trust this technology fully yet.

SEP

Re: qlogic failover & hp_rescan que?

Craig Gilmore — Mon, 06 Mar 2006 21:46:55 GMT

Hi,

hp_rescan has been tested in several different environments. It should be safe to run this tool in a production environment. I have run the tool on a system in production, to find an added LUN.

However, since you seem to be concerned, you can reboot the system to rescan the scsi bus. The tool was created to help avoid the reboot.

Regards,

Re: qlogic failover & hp_rescan que?

Alan_152 — Tue, 07 Mar 2006 10:54:20 GMT

re #2:

I've run hp_rescan with no problems on multiple prod environments. You could also reboot, or try the following script that I've found:

#!/bin/bash
# rescan_scsi_bus.sh
# Script to rescan SCSI bus, using the
# scsi add-single-device mechanism
# (w) 1998-03-19 Kurt Garloff (c) GNU GPL
# (w) 2003-07-16 Kurt Garloff (c) GNU GPL
# $Id: rescan-scsi-bus.sh,v 1.15 2004/05/08 14:47:13 garloff Exp $

setcolor ()
{
red="\e[0;31m"
green="\e[0;32m"
yellow="\e[0;33m"
norm="\e[0;0m"
}

unsetcolor ()
{
red=""; green=""
yellow=""; norm=""
}

# Return hosts. sysfs must be mounted
findhosts_26 ()
{
hosts=
if ! ls /sys/class/scsi_host/host* >/dev/null 2>&1; then
echo "No SCSI host adapters found in sysfs"
exit 1;
#hosts=" 0"
#return
fi
for hostdir in /sys/class/scsi_host/host*; do
hostno=${hostdir#/sys/class/scsi_host/host}
hostname=`cat $hostdir/proc_name`
hosts="$hosts $hostno"
echo "Host adapter $hostno ($hostname) found."
done
}

# Return hosts. /proc/scsi/HOSTADAPTER/? must exist
findhosts ()
{
hosts=
for driverdir in /proc/scsi/*; do
driver=${driverdir#/proc/scsi/}
if test $driver = scsi -o $driver = sg -o $driver = dummy -o $driver = device_info; then continue; fi
for hostdir in $driverdir/*; do
name=${hostdir#/proc/scsi/*/}
if test $name = add_map -o $name = map -o $name = mod_parm; then continue; fi
num=$name
driverinfo=$driver
if test -r $hostdir/status; then
num=$(printf '%d\n' `sed -n 's/SCSI host number://p' $hostdir/status`)
driverinfo="$driver:$name"
fi
hosts="$hosts $num"
echo "Host adapter $num ($driverinfo) found."
done
done
}

# Test if SCSI device $host $channen $id $lun exists
# Outputs description from /proc/scsi/scsi, returns new
testexist ()
{
grepstr="scsi$host Channel: 0*$channel Id: 0*$id Lun: 0*$lun"
new=`cat /proc/scsi/scsi | grep -e"$grepstr"`
if test ! -z "$new"; then
cat /proc/scsi/scsi | grep -e"$grepstr"
cat /proc/scsi/scsi | grep -A2 -e"$grepstr" | tail -n2 | pr -o4 -l1
fi
}

# Perform search (scan $host)
dosearch ()
{
for channel in $channelsearch; do
for id in $idsearch; do
for lun in $lunsearch; do
new=
devnr="$host $channel $id $lun"
echo "Scanning for device $devnr ..."
printf "${yellow}OLD: $norm"
testexist
if test ! -z "$remove" -a ! -z "$new"; then
# Device exists and we're in remove mode, so remove and readd
echo "scsi remove-single-device $devnr" >/proc/scsi/scsi
echo "scsi add-single-device $devnr" >/proc/scsi/scsi
printf "\r\x1b[A\x1b[A\x1b[A${yellow}OLD: $norm"
testexist
if test -z "$new"; then
printf "\r${red}DEL: $norm\r\n\n\n\n"; let rmvd+=1;
fi
fi
if test -z "$new"; then
# Device does not exist, try to add
printf "\r${green}NEW: $norm"
echo "scsi add-single-device $devnr" >/proc/scsi/scsi
testexist
if test -z "$new"; then
# Device not present
printf "\r\x1b[A";
# Optimization: if lun==0, stop here (only if in non-remove mode)
if test $lun = 0 -a -z "$remove" -a $optscan = 1; then
break;
fi
else
let found+=1;
fi
fi
done
done
done
}

# main
if test @$1 = @--help -o @$1 = @-h -o @$1 = @-?; then
echo "Usage: rescan-scsi-bus.sh [options] [host [host ...]]"
echo "Options:"
echo " -l activates scanning for LUNs 0-7 [default: 0]"
echo " -w scan for target device IDs 0 .. 15 [default: 0-7]"
echo " -c enables scanning of channels 0 1 [default: 0]"
echo " -r enables removing of devices [default: disabled]"
echo "--remove: same as -r"
echo "--nooptscan: don't stop looking for LUNs is 0 is not found"
echo "--color: use coloured prefixes OLD/NEW/DEL"
echo "--hosts=LIST: Scan only host(s) in LIST"
echo "--channels=LIST: Scan only channel(s) in LIST"
echo "--ids=LIST: Scan only target ID(s) in LIST"
echo "--luns=LIST: Scan only lun(s) in LIST"
echo " Host numbers may thus be specified either directly on cmd line (deprecated) or"
echo " or with the --hosts=LIST parameter (recommended)."
echo "LIST: A[-B][,C[-D]]... is a comma separated list of single values and ranges"
echo " (No spaces allowed.)"
exit 0
fi

expandlist ()
{
list=$1
result=""
first=${list%%,*}
rest=${list#*,}
while test ! -z "$first"; do
beg=${first%%-*};
if test "$beg" = "$first"; then
result="$result $beg";
else
end=${first#*-}
result="$result `seq $beg $end`"
fi
test "$rest" = "$first" && rest=""
first=${rest%%,*}
rest=${rest#*,}
done
echo $result
}

if test ! -d /proc/scsi/; then
echo "Error: SCSI subsystem not active"
exit 1
fi

# defaults
unsetcolor
lunsearch="0"
idsearch=`seq 0 7`
channelsearch="0"
remove=""
optscan=1
if test -d /sys/class/scsi_host; then
findhosts_26
else
findhosts
fi

# Scan options
opt="$1"
while test ! -z "$opt" -a -z "${opt##-*}"; do
opt=${opt#-}
case "$opt" in
l) lunsearch=`seq 0 7` ;;
w) idsearch=`seq 0 15` ;;
c) channelsearch="0 1" ;;
r) remove=1 ;;
-remove) remove=1 ;;
-hosts=*) arg=${opt#-hosts=}; hosts=`expandlist $arg` ;;
-channels=*) arg=${opt#-channels=};channelsearch=`expandlist $arg` ;;
-ids=*) arg=${opt#-ids=}; idsearch=`expandlist $arg` ;;
-luns=*) arg=${opt#-luns=}; lunsearch=`expandlist $arg` ;;
-color) setcolor ;;
-nooptscan) optscan=0 ;;
*) echo "Unknown option -$opt !" ;;
esac
shift
opt="$1"
done

# Hosts given ?
if test "@$1" != "@"; then
hosts=$*;
fi

echo "Scanning hosts $hosts channels $channelsearch for "
echo " SCSI target IDs " $idsearch ", LUNs " $lunsearch
test -z "$remove" || echo " and remove devices that have disappeared"
declare -i found=0
declare -i rmvd=0
for host in $hosts; do
dosearch;
done
echo "$found new device(s) found. "
echo "$rmvd device(s) removed. "

Re: qlogic failover & hp_rescan que?

klemerS — Tue, 07 Mar 2006 12:02:37 GMT

Hi.

I Think that i will reboot the machine.

2. where i can found docs about the qlogic?
Because i have already 2 sites that i see
the luns only on one HBA & its seems to work O.K. - But from my memory its wrong ,
so can someone point me to doc who axplain
this issue?

Thanks Alot to you Guys for your answers.

Re: qlogic failover & hp_rescan que?

klemerS — Tue, 07 Mar 2006 12:47:26 GMT

Hi.

i found A doc which say that the on the
other HBA you see nothing.

https://uimon.cern.ch/twiki/bin/view/PSSGroup/Multipathing

--------------------------------------------------------------------------------

Start of topic | Skip to actions
--------------------------------------------------------------------------------

-->

Multipathing with Qlogic HBAs - failover and LUN load balancing
This is a short guide to the configuration of multipathing with Qlogix HBAs for the physics database services at CERN. The key software and hardware components are: Oracle 10gR2, RAC, ASM, Linux RHEL 3, dual CPU Intel servers, SAN network, SATA disks in FC arrays (see also Reference Architecture). SAN and FC (2 Gb) infrastructure: Qlogic dual-ported HBAs (QLA2342), 2 x Qlogic SAN switches, dual-ported infortred FC arrays (A16F-S1211). Note: tested with RHEL 3 U6, QLA2342 driver=7.05.00-RH1-f0, firmware=3.03.11, bios=1.47. SANsurfer=2.0.30b69

Under the term multipathing, in this document, we refer to a SAN configuration for path failover redundancy and load balancing:

redundant path to access the storage arrays
failover at the LUN level for failed path from the sevrer to the SAN switch (improved HA)
load balancing at the LUN level (improved scalability and performance)

OS configuration files
Multipathing for Qlogic HBAs is configured via /etc/modules.conf, and /etc/qla2300.conf. Moreover it is important to check that both qla2300_conf and qla2300 kernel modules are loaded (ex: lsmod |grep qla23)

After LUN load balancing configuration the /etc/modules.conf files should contain the following entry:

alias scsi_hostadapter0 qla2300_conf
alias scsi_hostadapter1 qla2300
options qla2300 ConfigRequired=1 ql2xuseextopts=1 ql2xfailover=1 qlport_down_retry=1 ql2xretrycount=2

FC and SAN configuration

Change the PID number of the storage array controllers for each array use the same PID for both controllers.
Configure zoning in both SAN switches and make the storage arrays visible via both HBA ports of the linux servers.

Qlogic HBA configuration
The /etc/qla2300.conf configuration can be done using Qlogic SANsurfer for HBA (downloadable from the Qlogic website).

install the agent of SANsurfer for HBA on the nodes you need to configure
disable autostartup of the qlogic agent
chkconfig qlremote off
manually start and stop the agent using /etc/init.d/qlremote start or stop
Install and run the SANsurfer management console and connect to the node you want to configure
Don't run the node configuration wizard
Click the 'configure' icon
DEVICE menu
CONFIGURE
CONFIGURE ALL
FAILOVER
ENABLE ALL
LUN menu
ENABLE LUNS
ENABLE ALL
LOAD BALANCE
ALL LUNS
Save the configuration (default_pass='config')
Check the configuration in /etc/qla2300.conf and /etc/modules.conf.
/etc/init.d/qlremote stop
rmmod qla2300
rmmod qla2300_conf
modprobe qla2300

Test the stability of the configuration across reboot
Recreate the init ramdisk: mkinitrd /boot/initrd-2.4.21-37.ELsmp.img 2.4.21-37.ELsmp
reboot and check that the new configuration is being picked up

Post install - sample qla2300.conf configuration
This is a sample /etc/qla2300.conf configuration file for anode configured with 3 disk arrays, using dual ported HBAs on the server and the array. Note: extra new lines and comments have been added for clarity.

scsi-qla0-adapter-port=210000e08b1481b8\; -> WWN identifier of the first local HBA port (port 1)

scsi-qla0-tgt-0-di-0-node=200000d0230e0671\; -> WWNN: storage array controller (array N.1 of 3)
scsi-qla0-tgt-0-di-0-port=210000d0230e0671\; -> WWNP: storage array 1/3 controller port 1
scsi-qla0-tgt-0-di-0-pid=020cef\; -> ?
scsi-qla0-tgt-0-di-0-preferred=0000000000000000000000000000000000000000000000000000000000015555\;
-> command related to the activation LUN load balancing
scsi-qla0-tgt-0-di-0-control=00\;
-> storage array N.1 is bound to port 0 (but LUN load balacing makes for
the desired alternate access path)

scsi-qla0-tgt-1-di-0-node=200000d02350000e\; -> WWNN array N.2/3
scsi-qla0-tgt-1-di-0-port=210000d02350000e\; -> WWNP array N.2/3
scsi-qla0-tgt-1-di-0-pid=020e33\;
scsi-qla0-tgt-1-di-0-preferred=0000000000000000000000000000000000000000000000000000000000015555\;
scsi-qla0-tgt-1-di-0-control=00\;

scsi-qla0-tgt-2-di-0-node=200000d0231e0279\; -> WWNN array N.3/3
scsi-qla0-tgt-2-di-0-port=210000d0231e0279\; -> WWNP array N.2/3
scsi-qla0-tgt-2-di-0-pid=020fe8\;
scsi-qla0-tgt-2-di-0-preferred=000000000000000000000000000000000000000000000000000000000000aaaa\;
scsi-qla0-tgt-2-di-0-control=00\;

scsi-qla1-adapter-port=210100e08b3481b8\; -> WWN identifier of the first local HBA port (port 2)
scsi-qla1-tgt-0-di-1-node=200000d0230e0671\; -> WWNN: same as above, array controller (array N.1 of 3)
scsi-qla1-tgt-0-di-1-port=220000d0230e0671\; -> WWNP: storage array 1/3 controller port 2
scsi-qla1-tgt-0-di-1-pid=010cef\; -> ?
scsi-qla1-tgt-0-di-1-preferred=000000000000000000000000000000000000000000000000000000000000aaaa\;
scsi-qla1-tgt-0-di-1-control=80\;
-> port 1 is failover for storage array N.1

scsi-qla1-tgt-1-di-1-node=200000d02350000e\;
scsi-qla1-tgt-1-di-1-port=220000d02350000e\;
scsi-qla1-tgt-1-di-1-pid=010e33\;
scsi-qla1-tgt-1-di-1-preferred=000000000000000000000000000000000000000000000000000000000000aaaa\;
scsi-qla1-tgt-1-di-1-control=80\;
scsi-qla1-tgt-2-di-1-node=200000d0231e0279\;
scsi-qla1-tgt-2-di-1-port=220000d0231e0279\;
scsi-qla1-tgt-2-di-1-pid=010fe8\;
scsi-qla1-tgt-2-di-1-preferred=0000000000000000000000000000000000000000000000000000000000015555\;
scsi-qla1-tgt-2-di-1-control=80\;

Post install - sample /proc output
A sample from the /proc/scsi/qla2300/1 file is reported below. The list of SCSI LUNinformation on /proc/scsi/qla2300/2 should be empty. However, the last column in the list reports for each LUN the HBA port that is used. For example '0:0:81' means port 1, while '1:1:81' means port 2. In case of failover these numbers change accordingly (see also below):

SCSI LUN Information:
(Id:Lun) * - indicates lun is not registered with the OS.
( 0: 0): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,
( 0: 1): Total reqs 5, Pending reqs 0, flags 0x2, 1:1:81,
( 0: 2): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,
( 0: 3): Total reqs 5, Pending reqs 0, flags 0x2, 1:1:81,
( 0: 4): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,
( 0: 5): Total reqs 5, Pending reqs 0, flags 0x2, 1:1:81,
( 0: 6): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,
( 0: 7): Total reqs 5, Pending reqs 0, flags 0x2, 1:1:81,
( 0: 8): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,
( 0: 9): Total reqs 5, Pending reqs 0, flags 0x2, 1:1:81,
( 0:10): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,
( 0:11): Total reqs 5, Pending reqs 0, flags 0x2, 1:1:81,
( 0:12): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,
( 0:13): Total reqs 5, Pending reqs 0, flags 0x2, 1:1:81,
( 0:14): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,
( 0:15): Total reqs 5, Pending reqs 0, flags 0x2, 1:1:81,
( 0:16): Total reqs 5, Pending reqs 0, flags 0x2, 0:0:81,

Post install - sample failover log
In case of LUN failover the following information is displayed in the /var/log/messages file:

..... kernel: qla2x00: FAILOVER device 0 from 210000d0230e0671 -> 220000d0230e0671 - LUN 0b, reason=0x1
..... kernel: qla2x00: FROM HBA 0 to HBA 1
The failover is triggered only when the failed LUN is accessed, while the failback is triggered as soon as the failure is restored:

..... kernel: qla2x00: FAILBACK device 0 -> 200000d0230e0671 LUN 0b
..... kernel: qla2x00: FROM HBA 1 to HBA 0
History: Version 1, Luca Feb-2006

to top