2600 Stack problem

[Tony Barrett_2]

I have around 45 2600 series switches around our site, and a few of them are in stacks. There are two stacks that have >6 switches connected together, and for the most part this works ok.

The problem I'm noticing is that after switch 4 in the stack (i.e. 5th or 6th switches) I get large amounts of traps reported into NNM about loss of connectivity to the switch. Further investigation shows that in both stacks where there are >6 switches, the 5th & 6th switches show high CPU spikes (99%). This can last for quite a few seconds, then goes back to normal. I suspect during this spike, the switch does not respond to the master in the stack quick enough, so a false/positive trap is generated. Also, looking at the stats, on these specific switches, I'll often see the 'lowest' packet buffer as '0', with a high 'missed' number.

All switches are running f/w 10.45, but this also happened with f/w 8.76. I upgraded hoping the problem would get resolved. No users on these switches complain of any disconnection problems, but I guess if the CPU is spiking, or packet buffers are exhausted, this will cause performance issues.

There are no extra VLANS or unusual configs on these switches. LACP is disabled on the switch stack ports. As I said, I've only seen this problem on the stacks with >6 switches, and then only on the 5th or 6th switches in the stack. The smaller stacks of <=4 switches have no problems.

Anyone have any ideas?

[Matt Hobbs]

Are you using the 'stacking' feature Tony? (i.e. one IP address to manage them all?). Personally, I'm not a huge fan of that feature and if possible will always try and give my switches individual IP addresses and disable the stacking feature: 'no stack'.

Otherwise this does sound like an issue that could be reproduced in a lab environment, if you think it can be I'd open up a case with HP support.

[Tony Barrett_2]

Thanks Matt.

I'm not using the 'stacking' feature where a single IP address is assigned for management. Each switch has it's on management IP, and the lead switch (with core uplink) is designated commander. Each other switch in the stack is then 'joined' to the stack with manual assignment. They are not 'candidates' for any other stack.

This does allow me to connect to the web agent on the lead switch, and view the whole stack, and also certain switch settings are inherited by the members from the commander.

If I do a 'show stack' on the commander;

Stack State: Commander
Transmission Interval: 60
Stack Name: Stack1
Auto Grab: No
Number of members: 6
Members unreachable: 0

From the 5th switch member, 'show stack';

Stack state: Member
Transmission Interval: 60
Switch number: 5
Stack Name: Stack1
Member status: joined successfully
Commander status: Commander Up
Commander MAC Address: 00110a-d63800

If I look at the switch log on the 'commander' I regularly see the following (the MAC's are from the 5th and 5th switches in the stack);

W 12/18/07 02:14:47 FFI: Stack member 001185-36c600 lost.
W 12/18/07 02:26:41 FFI: Stack member 001438-f62f80 lost.

If I look on the switches that are being 'lost' from the stack, I see errors similar to the following;

W 12/16/07 15:35:41 system: Out of pkt buffers; miss count: 100535

Now, this is either a bug in the stacking feature, or a problem I can't find with the setup. I suspect a bug, so maybe HP need to investigate further.