BL460c + VC Flexfabric + iSCSI HBA not able to reach somes hosts

[ricdanger21]

Hi there,

I have a new C7000 with two VC Flexfabric modules, connected to a Cisco 4500 at the core, and 8 BL460c G7 blades.

I have two iSCSI storage vlans. Both are setup as vNet, though a shared uplink, on my VCs.

If I setup my BL460C G7with flexnics (non-hba) connected to the storage vNets I can connect and ping both my NetApp storage and my switches. I can also ping other hosts on the storage networks, even if they are connected to my 4500 or on the VC.

The strange thing is that If I setup  iSCSI FlexHBAs instead, I can only ping some of the hosts. I can ping my ESX servers connected to my 4500, I can ping hosts connected to the VC, and I can ping WIndows hosts just fine. But I can't ping my NetApp or my Cisco 4500 anymore. It doesn't matter if I setup the iSCSI details on virtual connect or through the emulex bios. It doesn't ping in either scenario.

To make it clear. If I setup one Flex-10 nic on the storage networks I can ping all my hosts and connect to my iSCSI targets. If I replace the flex-10 nic with one iSCSI hba (everything else is the same) I can't ping my NetApp storage or my default gateway (the 4500), and I can't connect to the iSCSI targets.

Any idea what can be the problem? It seems like a bug on the HBA.

Things I checked:

- My 4500 shows the correct MAC address for the iSCSI HBA IP address, on the Cisco 4500 arp table

- The MAC address shows assigned to the correct uplink on the Cisco 4500

Thanks.

ps: iSCSI sessions also do not work; this is not just ping not working

[Hongjun Ma]

it does look like some bug behavior if you can have full connectivity when configuring a regular NIC under the same iSCSI vnet.

Can you try the following on one blade to see

-set the server to not poweron at insert into the chassis.  this can be done from the iLO web administration under power settings.  it is a checkbox. then click submit. (you can also change this power-on setting in OA GUI instead of iLO)

-power off the blade server

-unassign server profile in virtual connect

-unseat the blade for 30 seconds to allow voltage to clear from iLO

-reseat the blade server

-reassign server profile

-boot server

If it doesn't work, try to also upgrade CNA firmware to latest 4.0.360.15(using .20 ISO). I don't think it's FW issue but just in case you have some old CNA firmware. After upgrade, try the above procedure again.

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=4324854&prodTypeId=3709945&prodSeriesId=4296125&swLang=8&taskId=135&swEnvOID=54

[ricdanger21]

Hi,

I'm already using the latest firmware on both VC, CNA, OA, Bios and iLO.

I also tried previous CNA firmwares, with the same results.
Reseating didn't help. Also, the same behaviour happens on all my BL460c G7.

Regards.

[Hongjun Ma]

thanks. unless others have any suggestions. I'll say please open a support case with HP for further troubleshooting.

The same reseat procedure worked for the other customer iSCSI connectivity issue. If you can share HP support case here or send me a private message, that'll be great. I'd like to track down the status of this issue.

[ricdanger21]

Some findings:

- If I use tunnelled mode, everything works.

- If I set the storage VLAN as native on the SUS, everything works

So it seems like some bug/incompat between the NetApp and Cisco added tags or dot1q header.

I opened a case and am currently waiting for HP to give me some feedback.

[cjscol]

Did you ever get this working?

I have a similar issue.  Using BL465c G7 blades.  Configure the FlexFabric to present iSCSI HBAs, cannot connect to the NetApp Filer via the 10GbE ports on the Filer however I am able to connect to the 1GbE ports on the Filer.

The main difference between the 1GbE and 10GbE ports is that I have VLAN tagging on the 10GbE ports.  Not sure if my issue is getting connectivity from the NC551i confgured as an iSCSI HBA to the 10GbE ports or if the issue is because I have VLAN tagging on the 10GbE ports.  If I get chance I will configure VLAN tagging on the 1Gb ports to see if that breaks them and also take the VLAN tagging off the 10GbE ports to see if that fixes them.

If I configure the FlexFabric to present standard Ethernet adapters instead of iSCSI HBAs and then run a software iSCSI initiator over the Ethernet adapters I can connect to the 10GbE ports on the NetApp even with it configured with VLAN tagging.

[ricdanger21]

I know what is causing this, but couldn't get it fixed.

The packets that don't work have a non 0 CoS value on the 802.1q header.

For some reason (bug), the Emulex iSCSI HBA function drops this packets when using mapped mode on VC.

A workaround is to use tunneled mode for the iSCSI networks. In that case, iSCSI HBA works great with non 0 CoS.

I've openned a case with HP, but they are yet to get to me after their first troubleshoot, almost 2 weeks ago.

[Darda]

Let me know the support case number and who in EPM is working on this.  You should also create a SI case.

[Darda]

This issue is identified as a CNA firmware issue in the following:

The frames within the same VLAN but different user priorities are considered into two different "conversation."  I have asked the NIC vendor to correct it.

Using a dedicated VC network should avoid the issue.

[Infra_Bas]

Just wanted to let you guys know,

I'm having the exact same problem with booting G7 blades (NC553i) with virtualconnect from ISCSI on Netapp filers.

G6 blades do not have any problems booting from ISCSI. Also can confirm that  running the iscsi vlan as native VLAN is a possible workarround. Tunnel mode isn't an option for me because I would have to change a lot of configuration in my network, while all g6 blades are working fine.

Got the newest firmware and also opened a support case for the bl460c g7 and virtualconnect, lets hope they can fix this soon.

Bas