Cisco 3020 causing c3000 fans going maximum speed

[chuckk281]

Al had a customer that was having thermal problem and needed some help:

****************************************************************************

I remember seeing a posting regarding this same topic:

A customer c3000 chassis with Cisco Catalyst 3020 has been experiencing heat related symptoms with interconnect slot 1, causing fans 2 and 5 to accelerate to 100% speed and all others are at 91% speed. CLI log output and chassis status confirm slot one is the cause:

Jul  8 12:24:45  OA: Temperature of I/O module in slot 1 is at alert level

Jul  8 12:26:46  OA: IO module in slot 1 temperature is normal

Jul  8 12:36:53  OA: Temperature of I/O module in slot 1 is at alert level

Jul  8 12:38:54  OA: IO module in slot 1 temperature is normal

Jul  8 12:50:01  OA: Temperature of I/O module in slot 1 is at alert level

Jul  8 12:52:02  OA: IO module in slot 1 temperature is normal

Jul  8 13:02:08  OA: Temperature of I/O module in slot 1 is at alert level

Jul  8 13:04:09  OA: IO module in slot 1 temperature is normal

Jul  8 13:14:15  OA: Temperature of I/O module in slot 1 is at alert level

Jul  8 13:16:17  OA: IO module in slot 1 temperature is normal

Jul  8 13:25:47  OA: Temperature of I/O module in slot 1 is at alert level

Jul  8 13:27:25  OA: Authentication failure for user Administrator on the OA se.

Jul  8 13:27:48  OA: IO module in slot 1 temperature is normal

Jul  8 13:27:58  OA: Authentication failure for user Administrator on the OA se.

Jul  8 13:29:24  OA: Administrator logged into the Onboard Administrator on the.

Jul  8 13:38:55  OA: Temperature of I/O module in slot 1 is at alert level

Field service replaced the original Cisco 3020 last week for this same reason, but that option does not seem viable again. Is this normal for this switch or could there be something wrong with thermal sensors?

****************************************************************************

Monty provided a very detailed answer for Al:

************************************************************************

The c-Class Onboard Administrator Thermal Logic for interconnect cooling supports two different thermal strategies:

1. Temperature Thresholds provided by the interconnect module
2. OA directly reads temp sensor on interconnect

For the Cisco 3020 – the first strategy is used as reported by the OA for this module – “Temperature Sensor Absent”

See interconnect information on the OA GUI or CLI – example from the marketing1.usa.hp.com enclosure below.

Information
Product Name	Cisco Catalyst Blade Switch 3020 for HP
Management IP Address	16.83.89.143
Management URL	http://16.83.89.143
User Assigned Name	Enc1_Bay3
Part Number	WS-CBS3020-HPQ
Spare Part Number	432904-001
Serial Number	FOC1042T0FH
Type	Ethernet
Manufacturer	Cisco Systems, Inc.
Temperature Sensor	Absent

c-Class Interconnect cooling requires that all unused interconnect bays are filled with interconnect blanks – or the PARSEC cooling as described in the c7000 and c3000 architecture tech briefs will not work properly and some interconnects may get hot.

There are three temperature thresholds reporting in strategy 1 for interconnect cooling: warm, alert and critical.

-          If the interconnect using strategy 1 reports the warm threshold is exceeded, the OA slightly increases fan speed in that zone – but the OA does not report any thermal issue, this is a normal part of OA thermal logic.

-          For the alert threshold exceeded – the OA will increase the fan speed to high and write a message in the OA syslog as in your email below.

-          The critical threshold exceeded will result in the OA immediately powering off that interconnect module – this is the same logic that iLO follows to protect the server in the event of server critical temperature threshold exceeded.

Please ensure that all empty interconnect bays and device bays have the blank panel installed.

Is it possible that the ambient temperature was above 35 degrees Centigrade when that interconnect reported alert temperatures?

***************************************************************************************

Good info from Monty. Does this help explain some of the cooling aspects of the c7000? Have you had similar questions? Let us know and we will help.

[blueeye]

Hello.

We have the same problem.

Problem appeared on May this year.

At begining of July the switch was poweroff for short time.

Everything was well but after one month high temperature problem is back.

I think it is software problem of Cisco switch.

After one month some logs may be full and it cause high temperature ... or something similar ...

Best regards ...

[blueeye]

Hello chuck.

Have you solved Cisco 3020 temperature problem?

We have the same problem.

Karol

[Manuel Boosch]

Hello,

i know this post is some years ago, but we had the same problem before some days in our C3000 Enclosure and we didnt find a solution in the forums.

The HP Support sent an advisory regarding this issue. Running 4 Fans at 61 % is a feature not a bug....

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/kb/docDisplay/?javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.endCacheTok=com.vignette.cachetoken&javax.portlet.prp_ba847bafb2a2d782fcbb0710b053ce01=wsrp-navigationalState%3DdocId%253Demr_na-c03614174-1%257CdocLocale%253D%257CcalledBy%253D&javax.portlet.tpst=ba847bafb2a2d782fcbb0710b053ce01&sp4ts.oid=4186428&ac.admitt...

Regards

[guestlistnyc]

The issue may lie in the blade population order and may not be Cisco 3020 specific. 3020 could just be a trigger for a problem that already there. How are you populating the blades? On c3000 you MUST populate the blades from the bottom bays up starting from bay 1. If you have blades in bay 8 or started in reverse order this may cause the issue. Also how many blades do you have? If less than 4 half height then only 4 Fan rule should kick in for Zone1. For Zone 1 (bays 1,5,2,6) and Zone 2 (bays 3,7,4,8), the 6 fan rule should kick in. Move the blades in the proper order and rearrange to populate blades from the bottom bays up. See if that fixed the problem and if it did then it had nothing to do with Cisco 3020.

See Technical White paper: Architecture and Technologies in the HP BladeSystem c3000 Enclosure, page 29. It says: Half-height servers should be populated in the following order: Device bays 1, 5, 2, 6, 3, 7, 4, 8. Full Height servers should be populated from bottom to top. Do not expect to receive this information from the HP support team. See attached and Good Luck...