HP3500 losing configuration and poor ping response

[Drew_38]

We've recently installed a 3500yl-24G (J8692A) running latest software K.11.33. We have two problems with it.

1) It frequently loses the ip rip information for a particular VLAN when making a change to the configuration file. In the example attached I have some ip rip commands for VLAN 226. Sometimes when I made a further change to the config file the rip entries vanish. At first I didn't beleive my eyes but it had done this about 4 times now which is annoying as I lose connectivity with the switch!

2) It suddenly responds slowly to pings. Normally I have a ping response of less then 1ms. When I send 100 or more pings (size 16byte) it responds a couple of time with 500 or 600 or 700ms, or even drops altogether. There are only about 50 devices connected via this switch via other HP 26xxx edge switches, so it's hardly overloaded. The previous device we had in this location responded ok so I'm sure there is not a physical problem with the fibre. I should also add we have at 5406zl running K.11.33 in a totally different building and that is also responding in the same poor way to pings. The 53xx and 34xx series switches we have connected elsewhere in a similar fashion don't have such problems.

Anyone else experiencing the same as we are? It seems to me that there is a problem with this release of software.

Thanks in advance for any help;

Drew

[Matt Hobbs]

With the RIP issue you're seeing, I'd recommend you open a case up with HP support if you can reproduce it easily.

For the slow pings, this is not unusual behaviour as the task that responds to pings addressed to the switches IP runs at a low priority. Normal traffic takes precedence.

I believe if you set a 'management-vlan' type it will improve the ping response reliability, however the management-vlan type is not routable so you have to take that into consideration before using this feature.

[Drew_38]

In reponse to Matt I should have added that users who are on the xxx.yy.226.0 subnet are also complaining about periodic 'slowness' and when they ping a server they are also getting occasional 600-800ms responses when from elsewhere on site 1ms approx is more common. So it's not just the switch which is responding to pings slowly there seems to be a more general issue.

Regarding the rip configuration thanks for the suggestion I'll raise a case with HP.

Would stil be curious if anyone else has these switches and is having problems though.

Drew

[Drew_38]

Hi all just to add further to my own post here. Regarding the rip configuration problem I describe above I have done a little more testing just to try and make sure I'm not going mad.

What is happening is that when I make a change (such as changing the switch IP number) to the 54xx or 35xx from the Menu then the vlan rip information is deleted. When I make the same change from command line it does not delete the rip information. I am guessing the Menu system rather stupidly deletes information which it can't configure itself (such as the rip info).

I've seen this before on the 53xx series: the Menu system is unable to make changes to the configuration properly. Makes me think HP really are not testing their software properly. It's very annoying; I wasted about two hours of my time earlier this week as I had to phyiscally visit a 35xx switch which I'd lost access to due to this.

Do HP offer prizes for people who find bugs? That'll cheer me up!

Drew

[Matt Hobbs]

Hi Drew,

HP do offer prizes but I found out today that this bug has already been reported so you don't get the prize! (just kidding about the prizes) If you haven't already done so, contact HP and make them aware of this thread. That way they should be able to get you new firmware quicker once it has been fixed.

Alternatively as you have already found a workaround you could just wait until this bug appears as fixed in the release notes.

Don't forget to assign points to any replies that you receive here.

Matt

[Drew_38]

Matt;

How did you find out the issue with the Menu system had been reported? Not that I mind reporting it as well (which I have done so that might speed up the fix slightly) but if there was a list of outstanding bugs which HP knew about at least we'd not be going mad trying to figure out what the problem is!

I realise HP might not want to publicise what is not 100% correct with their equipment but if I'd known it was an issue I'd not have done the change in the way I did and hence not wasted my time.

I've still an outstanding issue with the slow response users linked via the 3500 are getting, so I won't close this thread yet.

Drew

[Mohieddin Kharnoub]

HI
My friend Drew, i have some suggestions !!

1- Why don't you assing an IP Address for the Default-Vlan (Vlan1), since you've enabled IGMP on this vlan (The IGMP feature does not operate if the switch or VLAN does not have an IP address configured manually or obtained through DHCP/Bootp).

2- Have you tried to obtain the Premium license for the 3500 and 5300, so you can enable
IP PIM-dense.

I recommend you to check page 3-21 in this file: ftp://ftp.hp.com/pub/networking/software/6200-5400-3500-Multicast-and-Routing-Jan-06.pdf

Good Luck !!!

[Matt Hobbs]

Hi Drew,

I found out through a contact at HP. For any new issues like this your best bet is to see if anyone else has reported it here on the forums, otherwise with HP directly.

The issue was only reported a few days before you noticed it, so there is a good chance that if you do speak to HP - the person you speak to may not be aware of it and will need to speak to someone else to confirm it (which is why I suggested mentioning this thread). Otherwise they might have taken you through the whole escalation process which would be unnecessary at this point.

In regards to the second part of your issue (slow response), can you help to narrow it down any further? i.e, When it occurs, how long does it occur for? When it occurs is the CPU usage on the switch higher than normal? Is there anything in the switch event log? If using spanning-tree, are any topology changes occurring? Is there anything out of ordinary when you look through the port counters? Have you tried using anything to monitor the network like PCM+ Traffic Monitor?

Matt

[Drew_38]

Hi all;

Sorry for delays in replies to my query. I do appreciate the attention in here. I've been having to do things like send a show tech to HP in The Netherlands as they are trying to reproduce the problem (even though it's a known bug by HP). That's to do with my first problem with the 3500 menu system.

Anyhow Mohieddin the reason there is no IP number on the default VLAN is that there should be nothing connected to the default VLAN. I put ip igmp on this VLAN on there as that was in when I originally configured and I didn't take it out. We do have a premium license for this device; at one point we did enable pim dense and it didn't make any difference. But it's good to know we're trying the right sort of things.

For Matt some more good questions thank-you. Basically pinging a server from a device connected via the 3500 sending 20 or so pings the normal response of less than 1ms but then in a seemingly random fashion 3 or so pings will suddenly respond with 600, 700 or 900ms, or even drop. So there's bursts of poor connectivity. Pinging the same server from elsewhere does not cause the problem. And I get the same problem from the HP5400 I have in service which is linked via a totally different fibre. There's nothing in the log when this is happening, the CPU is just ticking along at about 13% hardly changes. Port counters no errors. Tried it with spanning tree off and on (we don't normally configure spanning tree on routers).

Not tried PCM+ traffic monitor I can load it up and give it a go. But I suspect it'll not show anything.

Fundamentally the switch is configured in the same way we configure all our layer 3 switches, 5300s, 3400s etc. It's just performing poorly in the way I describe (and so is the 5400 we have).

Is there anyone here who has a layer 3 3500 and it's working fine? I'm curious to know!

Drew

[Mike Satterfield]

Drew, while I didn't experience the config issue this time, I also had similar performance issues with release 11.33 on the 3500yl. In my case it was an upgrade from the 3400cl and 5300. Performance was so bad I had to pull the 3500s out and go back to previous gear.

The network seemed fine until it came under its normal daily load. Then, the ping times went to hundreds of miliseconds in a cyclical fashion. End-users experienced the same cycle with site to site traffic.

Procurve is still looking into the issue.

Mike