topic Re: 5412zl High CPU Usage in Switches, Hubs, and Modems

5412zl High CPU Usage

Aarón — Wed, 22 Jul 2009 10:17:31 GMT

Hi everyone!

I hope you guys can give me some advice. Right now our main 5400 is having _really_ high CPU usage:

87 percent busy, from 1801 sec ago
1 sec ave: 99 percent busy
5 sec ave: 99 percent busy
1 min ave: 84 percent busy

I opened a case with HP Support and the adviced me about having LACP enabled with non-using LACP devices and trying to disable it where it was not being used.

Right now we can not change anything in the configuration because we are expecting a huge load of preinscription traffic next week.

I just want to know if it could or not be LACP related or maybe a vlan loop or something I'm missing.

We are not using spanning tree and it doesn't look like we have a loop (interface port utilization doesn't exceed 20%) and we are not even using it as a router, it's just switching.

Any ideas? I attached the show-tech all and running configuration.

FW version is K.13.25

Thank you in advance

PS: It's been like this since I know about and we had to reboot it once (with a firwmare update) but it didn't help. It held with average CPU around 12 and a couple minutes later it raised up again to 99

Re: 5412zl High CPU Usage

cenk sasmaztin — Wed, 22 Jul 2009 10:24:01 GMT

hi Aaron
as for me your network loop

please write this command on switch

(config)# spanning-tree and enter

after command entry watch cpu usage

and plese say me result

Re: 5412zl High CPU Usage

Aarón — Wed, 22 Jul 2009 10:38:53 GMT

Hi Cenk,

right now I can't change anything. Enabling spanning tree would desynchronize our VMWare heartbeats and people around here would probably kill me.

I just took a 'show interface port-utilization' snapshot and the highest one it's around 18% and also the CPU oscillates. Sometimes it even reaches 20% but then it raises again.

I attach you the snapshot + the 'show tech all'

Re: 5412zl High CPU Usage

cenk sasmaztin — Wed, 22 Jul 2009 10:46:58 GMT

I can't open your attachment

please send me only sh tech print

Re: 5412zl High CPU Usage

Aarón — Wed, 22 Jul 2009 10:49:19 GMT

Here you go

Re: 5412zl High CPU Usage

cenk sasmaztin — Wed, 22 Jul 2009 11:07:51 GMT

Aaron you must have enable spanning tree on switch

please test enable spanning tree on switch after watch sh spanning tree commamd and send me sh spanning tree command print

Re: 5412zl High CPU Usage

cenk sasmaztin — Wed, 22 Jul 2009 11:13:42 GMT

I see any other wrong hardware config on your switch

on 5400 series switch each module slot connect with 28.8Gbps(fullduplex)to main board (backplane)

if you want connect 10Gbps port on 5400 or 8200 switch you must have install on each module slot only two 10Gbps module

you have four 10 Gbps module
you must be each two module install different slot

Re: 5412zl High CPU Usage

Aarón — Wed, 22 Jul 2009 11:27:17 GMT

Oh, you are right, I had no idea about the backplane limitation in 5400 series for each module...

I will unplug them and then test if it is that.

Anyway, right now we are using just two 10GbE link so it wouldn't affect the backplane, right?

Re: 5412zl High CPU Usage

cenk sasmaztin — Wed, 22 Jul 2009 11:37:07 GMT

as for me no

you can't use full 10x4 gigabit bandwith usage on this slot

if you can use all bandwith on four 10 Gbps module on one slot at the time occur backplane problem

Re: 5412zl High CPU Usage

Aarón — Wed, 22 Jul 2009 12:07:04 GMT

I tried unplugging the 10GbE transceivers that we are not using and CPU is still high. I can't apply spanning tree now, if I do the initial state for the switch ports is Blocked so it will disable the ports temporarily...

We can't afford that right now, as soon as we are done with 'preinscription' season I'll try it

Thank you very much Cenk

Re: 5412zl High CPU Usage

cenk sasmaztin — Wed, 22 Jul 2009 12:10:56 GMT

you are welcome Aaron