ports on 5308 and 2650 going off-line/on-line 3 sec intervals,

[jtilenius]

Any ideas would be appreciated,

We have a core 5308 and 10 access layer 2650 switches. Around 50 servers and 200 PC's. In a 24 hour period, various ports will go off and back on for a 3 second delay. We have hard coded the speed and duplex on the ports that are having collisions and duplex mis-patches. Upgraded all the switches to the latest version....HP J4899A ProCurve Switch 2650
Firmware revision H.08.53


I 06/08/05 07:04:20 ports: port 11 is now off-line
I 06/08/05 07:04:23 ports: port 11 is now on-line
I 06/08/05 07:04:42 ports: port 11 is now off-line
I 06/08/05 07:04:45 ports: port 11 is now on-line
I 06/08/05 07:33:58 ports: port 2 is now off-line
I 06/08/05 07:34:02 ports: port 2 is now on-line
I 06/08/05 07:34:21 ports: port 2 is now off-line
I 06/08/05 07:34:27 ports: port 2 is now on-line
I 06/08/05 07:39:52 ports: port 10 is now off-line
I 06/08/05 07:39:55 ports: port 10 is now on-line
I 06/08/05 07:39:55 ports: port 10 is now on-line
I 06/08/05 07:40:14 ports: port 10 is now off-line
I 06/08/05 07:40:15 ports: port 10 is now on-line
I 06/08/05 07:41:31 ports: port 8 is now off-line
I 06/08/05 07:41:34 ports: port 8 is now on-line
I 06/08/05 07:41:53 ports: port 8 is now off-line
I 06/08/05 07:41:53 ports: port 8 is now on-line

Thanks for the help,

Jeff

[Les Ligetfalvy]

Have you edited out recurrances from your post, or do they only toggle a few times and settle down? If they settle down, then it is a "normal" part of the bootup negotiation process. I say "normal" in a guarded tone as it may vary by NIC and/or OS.

If the toggling continues, look at power management settings and also look at the cable plant or power ground loops as a source.

[jtilenius]

Hey Thanks,

Well we went to the finance area where this is happening and hard coded all the NIC's to 100/full and changed it on the switch port to match. Also went into power management settings on the NIC's and turned this feature off, on maybe 4 people we swapped cables when they still had issues after we hard coded the speed and duplex. This was all recommended by HP. So now we still have this port toggles. We have a fluke meter which tells us the PC's connected to each port, we ask the users if they have been rebooting, all have been left on line. I read several of your posts and some helpful ideas, is this just a common occurance with HP switches that they can't seem to fix?

[Søren Fisker]

I've always believed this to be a technical issue that should be worked around in firmware.

I've been through the same suggestions as you have.

Even if Les's suggestions work, I don't really see the use for HP network equipment if we can't use it with XP power management, speed autodetect and we need state of the art cabling to run 100 Mbit/s.

I've been fighting HP's socalled support for 6+ months on this, so don't hold your breath...

Regards,
Soren

[Anton Pikhteryev_2]

Have exactly the same problem in the lab. Can't stand it! Will HP support comment on that?

[Jonathan Axford]

Just to join in on the action, I suffer exactly the same problem on a few of my 5300 series switches...

The logs are constantly full of port fluctuations the same jtilenius specified in the original post. Would be good to hear from HP to see what they think....

[SCOOTER]

Hello Jeff,

What kind of problems do you experience when this happens? All I see in the small piece of log that a node boots or reboots.

Does this "behavior" cause severe networkproblems or are you just annoyed that there are a lot of entries in the log?

Don't get me wrong here, I'm just trying to understand what happens?


Regards,

SCOOTER

[Søren Fisker]

HP support is useless in this field. I fought the danish network support about 6 months. They will keep saying that is power management, bad cabling, bad drivers and so on. And then they'll run you through disable IGMP, STP, LACP, driver upgrades etc. all of which won't make the slightest difference...
And since the supporters won't acknowledge this as a problem, they don't report it to the engineers and it'll never get fixed.
This really sucks because Procurves are great products...

And apparently the engineers don't read forums because this has been discussed sooooo often :(

Regards,
Soren.

[Manfred Arndt]

Hi Søren,

Thanks for the kind words about ProCurve products, but sorry to hear that support has not been able to help you resolve this adaquately.

This can be very difficult to diagnose, since there are so many things that can cause auto-negotiation problems, including bad cables, bad drivers/fw, bad nics...but you know this already.

Do you have a known configuration that causes this problem? The more specifics the better -- the engineers can only fix issues if we can reproduce the problem.

Does this only happen with specific clients? Have you verified that this problem also occurs with a known good patch cable?

Do you have a Fluke cable tester? If not, this is usually one of the best investments you can make -- not to say that this is your problem, but I can't tell you how often I have run into "marginal" cables in the field that seem to work intermittently.

Regards

[Manfred Arndt]

Soren,

Sorry about your mangled name -- cut and past changed the character set on me after posting.

[Søren Fisker]

Hi, Manfred!

I completely understand why cabling/drivers/nic/firmware/BIOS etc. should be checked first.

And disabling as many protocols as possible is next.

But at some point you've got to move on.

I've tested a couple of connections with a Fluke which says that everything is just fine :)

This occurs randomly on all ports and the machines are HP evo510/530, dc7100's, nx5000, nx7010's, Intel boards with onboards Intel nics, Ecs boards with onboard Realtek 8139 etc.
Some have old drivers, som new.

We tried latest firmware for the switches and tried downgrading.

I've sent several 'show tech all' and topology maps etc.

Unfortunately I'm no longer responsible for the network, and we'll be switching to 5308's shortly.

It appears that the 10.04 firmware contains a partial fix for the problem (http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=856664&admit=-682735245+1130761060626+28353475)

Regards,
Soren
Who also figured out not to use ø in the forum :)

[Les Ligetfalvy]

Soren,
You infer wrong from my post. I said the FFIs that accompany the link up/down have been greatly reduced, not the link up/down.

I should point out that I have been able to fix all my link up/down issues by replacing jumpers, newer drivers, or disabling power management on the NICs.

[Manfred Arndt]

Soren,

Jeanniard, Preston, and your postings seem to imply that this random port up/down is only an issue with the 2600 series switches and that the 5300 and 2800 series do not seem to have this problem.

Am I reading this correctly?

Regards,
Manfred

[Søren Fisker]

Les,

I know you didn't say that the port toggling was fixed. Should've more clear, sorry.

Manfred,

I've only seen it with 2650's, but we haven't got any 5308s yet and our 2824 is a serverswitch.

About power managemet:

I've heard and read about this many times, and I still don't see why power management should cause port toggling. And if Procurves don't work with power management it should have a "NOT designed for Windows" logo on the box...

I'll stop ranting now :)

[Les Ligetfalvy]

Well... on the power management issue, I think the only thing we can fault HP for is that they did not manage your expectations.

The switch simply reports what it sees. If power management turns off the NIC, the switch does not know why it was turned off, simply that it did.

If the syslog entries in PCM are an assault to your eyes, type '-line' in the filter, select all, and delete. You may want to give it a quick perusal though before deleting or you may miss indicators of problems.

[Manfred Arndt]

Hi Soren,

Windows has a power management capability that can disable the NIC. When the NIC is disabled for power management, due to no network activity, it drops the link to the switch. In this case, the link up/down events are expected behavior with any vendor switch.

If you are concerned about the link up/down events, you can disable the Windows power management as a workaround or to troubleshoot this further. Some NICs also support their own power management capability, so please also refer to the vendor NIC documentation.

If NIC power management has been properly disabled, then I would not expect the port to be going on/off-line and should be investigated further.

[Søren Fisker]

Well, I did say "rant" :)

Are you saying that a link down, up, down, up sequence is expected behaviour for a nic going to power save?

It's also worth noting that jtilenius' issue doesn't end up with a changed state ie. the link goes off/on.

If this is due to power management, then it seems to be a nic being turned off by power management and woken up again almost right away by the PC or the switch?

Regards,
Soren

[Bruce Campbell_3]

Our site has some Procurve 2650
switches in addition to
Cisco 2900/3500/2950/3550
and Extreme Summit 200 and 48si.

I looked at ports going up and
down on some groups of switches,
and saw the following in a 24
hour period:

make number number link
of of ups &
switches ports downs

Cisco 24 343 742
Extreme 16 109 1120
HP 46 257 1176

number of switches = number of switches with
at least one port link
state change

number of ports = number of ports which had
at least one port link
state change

link ups and downs = number of link state
changes

Client computers are a mixture of
XP, linux, self managed, AD managed,
etc.

Cabling is CAT5, 5e and 6.

In conclusion, there are alot of ports
going up and down throughout the day,
but nothing in what I saw would indicate HP Procurve is significantly different than other makers. The link state changes
I am seeing are almost certainly caused
by the client computers.

[tobias Wester]

I have the exakt same problem, that ports on my 2848 go offand back on.
first i get a

duplex mismatch kl:11:20
Excessive CRC/Alignment Errors kl 11:20
Late collisions (collisions detected after transmitting ~64 bytes) were detected on port 11:20

the por i talk about is connected to out FW.
Cables and duplex is ceckt.

lacp is of
flow controll is on

any one have a suggestion what to do.
when this happends we lost conecction to the FW (internet)

[Les Ligetfalvy]

This is a pretty old topic to dredge up given that it hardly relates to you issue.

First off, for a dedicated link to a FW, you really should not be relying on auto-detect. Hard code the proper settings on both sides. That said, my guess is you have either a bad jumper or ground loop problem.

You really should start your own thread. That way you can assign points to good answers and are more likely to get better answers when points are at stake.