CPU0: insufficient dynamic memory

[Steve-Thompson]

First the error occurs something like this!
%%%%%%%%%%% OPCOM 25-SEP-2006 23:56:20.22 %%%%%%%%%%%
Request 108 was canceled
CPU0: for a request of 4592 bytes
Console heap space will be automatically increased in size by 128KB
PID bytes name
-------- ---------- ----
00000000 28384 ????
00000001 23424 idle
00000002 800 dead_eater
00000003 800 poll
00000004 371328 timer
00000005 232128 powerup
000007E2 76864 entry
0000000B 11392 shell_1
00000022 2688 pgb0_fcint
00000006 2880 tt_control
000005E9 8928 ????
00000014 1056 ????
00000018 384 ????
00000015 512 ????
00000013 1760 ????
0000028E 576 ????
0000026D 128 ????
00000007 800 mscp_poll
00000008 800 dup_poll
00000012 2336 shell_0
0000000A 13952 ????
0000000D 13952 ????
00000010 13952 ????
etc...
----------------------------------------

The environment:
2 node clustered ES40
4GB
Redundant (2) Memory channel on each node, <1Km distance between nodes.
SAN with HSG "dual channel" on each node, 4 paths to each disk.
All disks shadowed.
VMS 7.2-2 with patches.


The system does a spontaneous reboot, with VMS running ... it would appear to be an automatic effect of the expansion of "console heap space".
The variable "heap_expansion" is related to the installation of a say, a SmartArray (which which DONT have...)

Why did I say "with VMS running"?
According to the ES40 release notes, this "heap expansion" should occur at the console level...

The "crash" can happen every 2 or 3 days
We've varied the bios parameter "heap_expand" as being 2M or NONE but we still get the crash.
We believe the bios variable "heap_expand" should be NONE, which it has been, however we still get the error.

The other culprit could be VMS dynamic memory, this however is large and no expansions occur during normal processing.

Any ideas
Steven

[Duncan Morris]

Hi Steven,

although you have all the current 7.2.-2 patches, you do not indicate what firmware version you are running.

Judging by the age of the O/S, I suspect that you might have quite an old firmware version as well. V7.2 is current, whilst V6.2 dealt with a heap expansion problem associated with wwidmgr.

Were you able to determine what Request 108 was refering to? An examination of the old operator log after the reboot should help.
At least you would know what was triggering the heap expansion. This could be a driver other then the Smart Arrays.

Duncan

[Steve-Thompson]

The OPCOM message trail led to a backup process, handled by SLS software, which while not discounted doesnt convince us!

The firmware, yes, 6.2 does mention this as a bugfix. However we have 6.9 installed and we're putting in 7.2 tomorrow morning.

The reasoning we have at this moment (Fri PM), because there's no valid dump, is that VMS is getting killed by a HW fault, which is "resetting the console" and we're seeing the console heap exansion that occures AFTER the system "crash" ie. It's the second INIT that we see on console, not the first.

Lets see what happens this weekend!
More Monday!

[james_a_wells]

In my opinion the minimum firmware version software for 7.3-2 would be 6.5 (most stable) although I would recommend upgrading to at least 6.9

I have a three node ES45 cluster
running V7.3-2 with firmware of 6.9
and HSG80 controllers.

[Steve-Thompson]

Very odd. We havent had a failure for over 2 weeks!
I'll close this for now....