iSCSI FlexHBA timeout settings

[BeltwayBenny]

Hi Folks,

After many many years of using Qlogic iSCSI HBAs/Mezz in our HP servers we've started to transition to the FlexHBA.  Historically, we've set many timeout settings on each server that connects to our iSCSI SAN:

[All servers]

HKLM\SYSTEM\CurrentControlSet\Services\disk\TimeOutValue = xxx

[Servers with Qlogic HBA/Mezz]

IP_ARP_Redirect  - 100% required with our SAN

KeepAliveTO - Allows the HBA to wait during SAN failovers before sending an error to the driver stack

ExeThrottle  - Queue depth

MaxRequestHoldTime was set on each Windows OS that had an HBA.

So based on all of this info how are these settings set in the FlexHBA?  I did not see anything in the Emulex OneCommand manager.

Thanks,

Ben

[Hongjun Ma]

there is one timeout setting specified in VC iSCSI cookbook. You can find the link off my blog.

I'll do some research and let me know if there are other settings and let you know if I find out anything.

[BeltwayBenny]

Thanks, I did find that in the Emulex OneCommand there is a 'Host Driver Paramters' tab for the entire HBA but when I go to that tab all the settings are unavailable and the 'Installed Driver Type' field shows 'n/a'.  It seems one of the paramters is ETO (Extended Timeout Value) but I can not set it globally. 

Have not seen anything about ARP redirect in the OneCommand docs...  For VMware this might work 'esxcfg-hwiscsi -a allow vhmba#' but have not see anything for Windows, in Windows that is typically done inside the HBA.

Ben

[Hongjun Ma]

this is some reply I got from Emulex.

############

We have 2 iSCSI timers.

ETO which stands for Extended Timeout. Setting this timer will cause the driver to hold the outstanding IO for the time period set. ETO timers are started when the driver detects that the target is not responding to I/O requests or there is an explicit TCP notification (FIN/RST) from the target that the service is unavailable.

There are 2 ways of setting ETO values. One is during driver load time via the registry or insmod depending on the operating system (Parameter name is eto, refer to the steps below for information on how to configure this parameter). This will set ETO value for all iSCSI targets connected via OneConnect HBA.

Another way to set ETO is via the Emulex OCM utility. Once a target is connected, you can go to the specific target and set its ETO values. This will set the ETO value for that target alone. (Different targets can have different ETO values)

Supported values for this parameter in Windows are: Min = 0, Max = 3600, Def = 90

Supported values for this parameter in Linix are:         Min = 0, Max = 3500, Def = 30

LDTO which stands for Link Down Timeout. Setting this timer will cause the driver to hold the IO for the time period set when the  link is down between the adapter port and the port it is connected to. LDTO timer will start running the moment the link is lost on the HBA side. LDTO timer will be stopped when the link comes back up again.

LDTO is a driver load time configurable parameter on all operating systems.

On Windows, the LDTO value can be setup in the registry as follows

a.            Locate HKLM\System\CurrentControlSet\Services\be2iscsi\Parameters\Device\DriverParameters

b.            Edit DriverParameters (double click on DriverParameters) and enter (note the semicolon at the end)

ldto=n;

c.             Reboot the system

On Linux, the LDTO value can be setup during insmod time as follows insmod be2iscsi.ko ldto=n

Supported values for this parameter in Windows are: Min = 0, Max = 3600, Def = 20

Supported vlaues for this parameters in Linux are:      Min = 0, Max = 3600, Def = 20

[BeltwayBenny]

Hi, thanks for the info.

Based on your info I found additional driver info here: http://www-dl.emulex.com/support/windows/windows/250007/windows_drivers.pdf

We'll perform sometesting of the global registry paramers and I'll report back on my progress.

Ben

[BeltwayBenny]

Update. 

Using the latest SPP at the time of my last post we found the drivers to be unstable.  When testing network failover I (via the virtual connect) blackholed the VLAN on LOM1:b, the blackhole set the vlan # to 900 without affecting the link state of LOM1:b.  When the VLAN # was set to the original VLAN # the server (Windows 2008 R2, fully patched) BSOD'ed.  Repeated on a second server.

I was not expecting much help from HP support and concerned that when Googling FlexHBA issues there is very little information (for example, Google "FlexHBA Timeout" and this post is the first hit) about FlexHBA we decided to implement S/W iSCSI with Windows Firewall.  There are far more companies running S/W iSCSI than FlexHBA, I like to be in the sweet spot.

Ben

[Hongjun Ma]

thanks for the update. In regard to blackholing issue, how did you know vlan number is changed to 900? what's the vlan number originally? I'm assuming you have "smartlink" enabled when doing failover.

[BeltwayBenny]

The blackholing was done inside the VC domain (the chassis).  I created a new Ethernet Network (vlan 900) and then on LOMx:B changed from my iscsi Ethernet Network to the blackhole.  If testing went better I would have done similar tests on the upstream nexus 5K switches. 

Ben