topic Re: One-sided link down Cisco - HP GbE2 Switch in BladeSystem - General

One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Thu, 23 Sep 2010 10:37:09 GMT

Hi all,

I'm running into some problems connecting my GbE2 Switch to a Cisco 6506 2-system VSS setup.

The Blade side link is UP, while the Cisco side link remains 'notconnect'.

It's a change to a tagged setup with etherchannel link.

I've added the configuration of Switch A and some Cisco output (with LACP etherchannel and last summary normal etherchannel).

No matter which setup I try the link remains not connected.

What I've tried.
- reconfig from normal etherchannel (mode on) to LACP (mode active) on the cisco
- changed hardware: SFPs/cabling/fysical connection from switch 1 to switch 2 of the VSS system/changed uplink from sw A to sw B
- tried single link uplink, no etherchannel
- re-entered 'no shutdown' on the cisco interfaces

But no help there, the cisco side stayed notconnect.

Any help is appreciated!

Re: One-sided link down Cisco - HP GbE2 Switch

Pieter 't Hart — Fri, 24 Sep 2010 09:41:47 GMT

" switchport nonegotiate" missing on port 2/2/10 ?

what happens if you only connect one port of the configured channel?
what messages occur in the log of the switches?

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Fri, 24 Sep 2010 09:55:15 GMT

I have only tried 1 physical connection.
So 1 fiber from 1 bladeswitch to 1 coreswitch:
1/2/10 OR 2/2/10.
Both react the same way.

I have an small update though.

I configured port 21, which was in use before the reconfig in an uplink to an other enclosure (so that port worked) as a part of the trunk 2 as well.
Connected that one to port 1/2/10 of a cisco core: not connected
removed that port from the port-channel on the cisco, et voila: trunked and working:
blade trunk -> cisco core no-etherchannel

tried the same cable to the other port/core 2/2/10: not connected.

reconnected that same cable to the 1ste core again to get it working again: notconnected!

On all situations the bladeswitch notified as link going up on physical connection of the cable.

And it hasn't worked since.

Re: One-sided link down Cisco - HP GbE2 Switch

Pieter 't Hart — Fri, 24 Sep 2010 11:18:25 GMT

I don't see any reference to LACP in your GbE2 switches config
The default value is off

/cfg/l2/lacp/port mode off|active|passive

Re: One-sided link down Cisco - HP GbE2 Switch

Pieter 't Hart — Fri, 24 Sep 2010 11:32:14 GMT

the manaul also display an "apply" command

>> # /cfg/l2/trunk 5 (Select trunk group 5)
>> Trunk group 5# add 19 (Add port 19 to trunk group 5)
>> Trunk group 5# add 20 (Add port 20 to trunk group 5)
>> Trunk group 5# ena (Enable trunk group 5)
>> Trunk group 5# apply (Make your changes active)

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Fri, 24 Sep 2010 12:29:21 GMT

LACP of on the blades? ok! that's new info. I didn't read about having to enable it in any HP document I came across.

I will try that after the weekend.

Of course I did do an apply to make the changes active.

Thank you for thinking with me!

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Tue, 28 Sep 2010 11:27:36 GMT

Hi Pieter,

I've only just been able to check the LACP settings.

Unfortunately it doesn't seem to be an option in my switch.
There is no l2 nor lacp sub menu.

My current switch hardware + software:

HP ProLiant BL p-Class F-GbE2 Interconnect Switch A
sysName: DNB-ENCL01-SW01
sysLocation:
RackId: ******
RackName: Rack1
Enclosure: ******
EnclosureName: Enclosure1
Slot: A

Switch is up 4 days, 2 hours, 8 minutes and 41 seconds.
Last boot: 2:38:14 Fri Sep 24, 2010 (reset from console)

MAC address: 00:0e:62:33:34:00 IP (If 1) address: ******
Revision: 0A
Switch Serial No: ******
Hardware Part No: 336181-001 Spare Part No: 321148-001
Interconnect Module Part Number : 283193-B21 Spare Part Number: 321146-001
OctalFC Module Part Number : 321745- 21 Spare Part Number: 321145-001
Software Version 2.1.0 (FLASH image1), active configuration.

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Tue, 28 Sep 2010 11:30:20 GMT

I've used the following document to make my changes. It doesn't refer to lacp settings on the bladeswitch though:

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00457828/c00457828.pdf

Re: One-sided link down Cisco - HP GbE2 Switch

Pieter 't Hart — Wed, 29 Sep 2010 08:03:20 GMT

it doesn't refer to LACP but to IEEE 802.3ad trunking.

I miss step 3 in your config.

3. On both Cisco 6509 switches, enable 802.1Q tagging (VLAN trunking) on all ports connected to both GbE2 Interconnect Switches per the command:
6509# set trunk / nonegotiate dot1q 1-2

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Wed, 29 Sep 2010 08:25:10 GMT

I could try setting "switchport nonegotiate" on the port-channel on the cisco

the exact frase cannot be used as the document refers to CatOS config and ours is IOS based config.

Re: One-sided link down Cisco - HP GbE2 Switch

Pieter 't Hart — Wed, 29 Sep 2010 09:41:40 GMT

I've got a HP CLI reference from 2007 where example output allready shows "Software: Version 3.2.0".

so your "2.1.0" seems a little behind.

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Wed, 29 Sep 2010 09:44:54 GMT

it is, it is.

Prob. 2004.

I got a case running for this @ HP as well but haven't had any response yet.

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Wed, 29 Sep 2010 09:56:31 GMT

As I've got some servers semi-active on that blade chassis, I'm currently replacing my config to try the nonegotiate port-channel setting first.

If that doesn't succeed I'll look into an firmware update.
I've downloaded the latest now (3.2.12.0).

But as it's quite a big difference I'm a bit hessitative to upgrade now, and perhaps have a saved config which doesn't apply at several points anymore.

I'll post my findings here.

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Wed, 29 Sep 2010 10:44:40 GMT

the nonegotiate option doesn't seem to cut it.

see attached file for info

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Thu, 30 Sep 2010 08:33:32 GMT

For future reference by anyone reading this.

I have just upgraded firmware and boot image
old: boot 2.1.0
old: firm 2.1.0

actions:
firmware to 3.1.0
boot to 3.2.0
reset switch
firmware to 3.2.0
reset switch
firmware to 3.2.12
boot to 3.2.12
reset switch

old config is still working *pfew*

Next step: testing the new config.

Re: One-sided link down Cisco - HP GbE2 Switch

Pieter 't Hart — Thu, 30 Sep 2010 08:49:35 GMT

did you test if the l2 submenu is present now?

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Thu, 30 Sep 2010 09:01:18 GMT

It is!

LOTS of new options...omg!

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Thu, 30 Sep 2010 12:38:09 GMT

It's working!

Although it probably had nothing to do with the firmwareversions.

It was (at least part) a Cisco side misconfig.

The port-channel had storm-control specifically configured @ 10%.

The physical port had not.
Unfortunatly that meant it was set to standard with is stormcontrol @100%.

So stormcontrol mismatch, oops!

I will post the config from both here lateron

Re: One-sided link down Cisco - HP GbE2 Switch

Ivo_W — Mon, 04 Oct 2010 10:50:44 GMT

I promised to post the end config here for others to profit from.

Turns out it was a two-part problem (isn't it always?).

The link between the Cisco 6506 (in VSS mode) with port-channel enabled, wouldn't show as up or connected,
while the HP Blade Switch GbE2 would show as up.

Only when the HP Blade SWitch's firmware was up to date I had some results:
The portchannel's status changed into 'suspended' mode, indicating that ther was a link!

The suspended mode was due to a misconfig: stormcontrol on the interface was not set, and thus 100%.
Storm control on the portchannel was set to 10%, and hence a misconfig.

The log showing the misconfig:
#sh log
...
*Sep 30 11:12:29.587 GMT: %EC-SW1_SP-5-CANNOT_BUNDLE2: Gi1/2/10 is not compatible with Po31 and will be suspended (Bcast Supp. Level of Gi1/2/10 is 100.0%, Po31 is 10.0% )
...

Config of the portchannel:
interface Port-channel31
description HP Blade encl 3 SW A
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,100,200
switchport mode trunk
switchport nonegotiate
flowcontrol receive desired
storm-control broadcast level 10.00
storm-control multicast level 10.00

Config of the interface(s):
interface GigabitEthernet1/2/10
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 1,100,200
switchport mode trunk
switchport nonegotiate
storm-control broadcast level 10.00
storm-control multicast level 10.00
channel-group 31 mode on

#sh etherchannel 31 summary
Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
31 Po31(SU) - Gi1/2/10(P) Gi2/2/10(D)

(1 port down as it is not fysically connected yet)

Port 20 and 22 are the trunk ports on the Blade Switch side:
/c/port 20
tag ena
/c/port 20/gig
fctl both
speed 1000
mode full
auto off
/c/port 22
tag ena
/c/port 22/gig
fctl both
speed 1000
mode full
auto off

And VLAN spanning tree and trunk info:
/c/port 20
tag ena
/c/port 20/gig
fctl both
speed 1000
mode full
auto off
/c/port 22
tag ena
/c/port 22/gig
fctl both
speed 1000
mode full
auto off

Status info, showing the cross trunk up (Trunk 1) and the uplink trunk up (Trunk 2), 1 port fysically not connected yet:
>> Layer 2# trunk
Trunk group 1: Enabled
Protocol - Static
port state:
17: STG 1 forwarding
18: STG 1 forwarding

Trunk group 2: Enabled
Protocol - Static
port state:
20: STG 1 DOWN
22: STG 1 forwarding

I hope this is of use to anyone running into similar problems.
Thanks to Pieter for thinking with me on this problem!

Re: One-sided link down Cisco - HP GbE2 Switch

Pieter 't Hart — Mon, 04 Oct 2010 10:54:16 GMT

thanks for reporting back Ivo,
regards,
Pieter