Switches, Hubs, and Modems
cancel
Showing results for 
Search instead for 
Did you mean: 

Swicthes stopped updating PCM2.0+

SOLVED
Go to solution
Jonathan Axford
Trusted Contributor

Swicthes stopped updating PCM2.0+

Hi all,

I have been running PCM2.0+ for a while now and it has been working fine.
All of a sudden, Some of the switches have stopped responding and i can no longer update the software on them from the management server.

There are 3 5308 switches, 1 4108 and 2 9308 switches that are experiencing this problem.

You can scan for configurations without any problems, As soon as you start the software update wizard it times out on the first part.

Anyone come across this before or have any ideas???

Cheers

Jonathan
Where there is a will there is a way...
12 REPLIES
Preston Gallwas
Valued Contributor

Re: Swicthes stopped updating PCM2.0+

Jonathan, I'd be interested to know what the CPU usage and processes are on your machine (and specs) when it times out.

I have a long standing case (since Sept-Oct) with HP open about extremely high CPU Usage from Mysqld-max-nt.exe in PCM 2.0 (and 2.1)+ when performing some operations. Unfortunately, no resolution has come yet.

Devon Dawson
Advisor

Re: Swicthes stopped updating PCM2.0+

Jonathan,

While I dig into this can you let me know if the switches in question successfully pass the "Test Communication Parameters" tests available via the right-click menu items and/or the toolbar for a specific device.

I'm wondering if perhaps the devices have too many telnet sessions left open or perhaps the communication parameters have changed (within PCM, the switch or both).

If you could run this test and let us know the results that would be very helpfull.
Jonathan Axford
Trusted Contributor

Re: Swicthes stopped updating PCM2.0+

Hi,

The process usagee is very low on the box, It stays at around 4% and occasionally spikes to 30% so i don't think that can be the problem.

I have about 48 switches currently being managed by the server, All of these will allow you to schedule a software update.

I have checked and double checked the Communication parameters in PCM. The comms parameter test is successful on all accounts.

I have upgraded to version 2.1 to see if this has any effect.

I hadn't thought of open telnet sessions etc. I will check this now...

Where there is a will there is a way...
Les Ligetfalvy
Esteemed Contributor

Re: Swicthes stopped updating PCM2.0+

When I first started playing with Procurve switches and PCM, I did ot realize that improperly exiting a telnet session would not release it at the switch and prevent PCM from updating.

I have since learned to always do a proper logoff from a telnet session and to not linger in a session too long.
Sergej Gurenko
Trusted Contributor

Re: Swicthes stopped updating PCM2.0+

Customers reports that there is a problems with Q-trunks after upgrade. Recent HP firmwares support CDP in passive mode only. In some configurations half automatically discovered LACP/PaGP/1dotQ stops working after CDP disable.
Devon Dawson
Advisor
Solution

Re: Swicthes stopped updating PCM2.0+

Jonathan,

The engineers responsible for the software update tool in PCM have come back to me with a few clarifying questions:

1)When you say that the devices "stop responding" I assume you mean the are not responding to the first step of the software update wizard. Is that accurate and if not in what other ways do the switches stop responding?

2)Are all of your devices of the types you mentioned (5308,4108,9308) failing or are the ones you mentioned a subset of what's on your network. Just wondering if this is a class-wide problem (that is all 5300's you have deployed) or device specific (that is, 3 of your N 5300's).

3) Have you tried increasing the timeout value used by PCM for CLI and SNMP access? You can configure this by selecting the device and launching the "Comm. Parameters in PCM" tool. If the first step of the Software Update tool fails it is usually due to a timeout or mismatched communication parameters.

4) If possible, can you try rebooting one of the devices in question and letting us know if that changes anything. This will clear out any lingering telnet sessions but from your report that the communication test was successful I don't think this is likely to be the problem.

4)After restarting your management server can you please try the following steps:
- Rediscover one of the devices that is failing
- Test the communication parameters again to make sure everything is successful
- Try scanning the configuration on the device using the Configuration Manager "Scan" tool.
- Lastly, try running the software update wizard once again on the same device.

We are very interested in what you experience in each of these steps as something does seem to be amiss here. Once you have tried this can you let us know how it went and attach the file:

PNM\server\logs\FWU-server.log

This should give us a view into what is going wrong.

Sorry you're running into this.
Jonathan Axford
Trusted Contributor

Re: Swicthes stopped updating PCM2.0+

Hi, Thanks for the reply, Here are the answers to your q's....

1)That is correct, they timeout on the first stage of the Software update wizard.
I can Still successfully scan for new configurations.

2)The devices that are failing are just a subset of what we have here.
4 out of 5 4108gl switches fail the update wizard.
4 out of 11 5300XL switches fail the update wizard
2 out of 4 9300 Switches fail.

3) I have tried, After you suggested it, increasing the timeout values configured in PCM. This has had no effect. I have checked and rechecked the communication settings and they are definately right.

4) I will try restarting one of the switches as soon as i can. Can't at the moment because all hell will break loose!

4)I have deleted one of the problem devices, Restarted the server and manually rediscovered the device. I reset the comms parameters and still no joy.

I have attached the requested log file.

Thanks for your help,
Where there is a will there is a way...
Mohamed Hamedi
Respected Contributor

Re: Swicthes stopped updating PCM2.0+

OK, I think I found a way for you to eliminate the Telnet release issue that might be causing the Wizard to timeout. This should stop SWU from doing a Telnet into the device.

We can try this with the 5300 you have.
PNM\server\config\devconfig\sw5300.oid; open this file and change the variable that indicated something like the number of OS Image versions.
To be honest I am trying to do this from memory because I donâ t have access to one these files. It should have the number 2, change that to 1. You'll need to restart the PCM server.

See if the Wizard now moves to the next step.
If it doesn't timeout then you can bet that it was some sort of Telnet issue.

Again I must stress that you are doing this on your own risk, you should never mess with application files blah blah blah.
Mohamed Hamedi
Respected Contributor

Re: Swicthes stopped updating PCM2.0+

Hi, just wanted to ask, have you checked that these devices have their default Telnet session set to CLI and not Menu?

Looking at the log, you can see that the password for telnet is invalid, can you try and use the CLI wizard and see if that works? My guess it should also timeout.
Johnson Dantis
Occasional Visitor

Re: Swicthes stopped updating PCM2.0+

Hi Jonathan,

I am Johnson Dantis working at ProCurve, and currently looking at the problem you had mentioned.

As per one of the suggestion given by Mohamed Hamedi to use CLI Wizard. Please let me know have you tried it. Also Please let me know the results of it. From the Software Update logs you had sent, it looks to me as if the CLI authentication parameters within PCM and Device don't match resulting in timeout in Software Update wizard.

In such case there is equal probability that CLI Wizard also should timeout if the CLI credentials is a mismatch. Please let me know the results once you have tried it out.

Also I request you to attach the following log files after performing the following steps on the switch were you encountered the Software Update timeout problem:

1)Delete the Device
2)Rediscover the Device using Discovery Wizard
3)Verify the communication parameters using Device Manager.
4) Run the CLI Wizard under "Config Manager"
5) Run the Software Update wizard.

After that please attach the following log files:

1) Device Manager log (\\server\logs\DM-DevMgrServer.log)
2)\\server\logs\CS-out.log
3)\\server\logs\CS-err.log
4)\\server\logs\FWU-Server.log

Sorry for the inconvinence.

Thanks,
Johnson
Jonathan Axford
Trusted Contributor

Re: Swicthes stopped updating PCM2.0+

Hi, Thanks for the replies, I have been away for a few days but i should get round to trying all of these suggestions out now......

:-)
Where there is a will there is a way...
Jonathan Axford
Trusted Contributor

Re: Swicthes stopped updating PCM2.0+

Hi Guys, Thanks for all of the help.

I have managed to get all of the swicthes working now.

A few of the switches had the 'operator' password and username set to the same as the 'Manager' settings.
As soon as i took this off the switches updated straight away. I have experienced this before with a 9300 switch. In that case someone had configured both operator and manager passwords the same so you could only ever get operator level access.

Thanks again for the help.

:-)
Where there is a will there is a way...