Galaxy and DECram discs in shared memory (V8.3)

[Colin Butcher]

Hi,

I've set up an AlphaServer 4100 with 6.5GB memory (2+2+2+0.5) and two partitions (2GB + 2GB +2.5Gb shared). Boots & runs OpenVMS ALpha V8.3 (with all current patches at of end April) fine.

I want to set up a DECram disc in shared memory, but am having trouble mounting the device from the other instance.

Interestingly enough the DECram IVP fails, so I wonder if this might just be a bug?

The DECram IVP runs fine on both instances in LOCAL mode.

It fails on both instances when run in SHARED mode.

Here's the output on the first instance:

SYSTEM on XDGLX1 $ @sys$test:decram$ivp shared
Begin DECRAM IVP ...
%MOUNT-I-MOUNTED, MDS9999 mounted on _$99$MDS9999: (XDGLX1)

Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
$99$MDS9999: (XDGLX1) Mounted 0 MDS9999 219 1 1
Number of difference sections found: 0
Number of difference records found: 0

DIFFERENCES /IGNORE=()/MERGED=1-
SYS$COMMON:[SYSEXE]MDMANAGER.EXE;1-
MDS9999:[000000]MDMANAGER.EXE;1
%SYSTEM-F-BADPARAM, bad parameter value

From SYS$COMMON:[SYSMGR]MDRECOVER.DAT
Disk $99$MDS9999 Size 250 Label MDS9999
Shared memory, nopersist, noserve, nowritebuffered.

SYS$GALAXY.EXE loaded and running.
SYS$MDDRIVER.EXE loaded and running.

End DECRAM IVP ...
SYSTEM on XDGLX1 $

SYSTEM on XDGLX1 $ @sys$test:decram$ivp shared
Begin DECRAM IVP ...
%SYSTEM-F-BADPARAM, bad parameter value
%SYSTEM-F-DRVERR, fatal drive error
SYSTEM on XDGLX1 $

Here's the problem command (with VERIFY enabled):

$ decram create disk mds9999/capacity=250/memory=shared/noserve/nopersist
%SYSTEM-F-BADPARAM, bad parameter value


Trying to set the capacity to 0 gets the same response (bad parameter value).

The device remains mountable / dismountable on the primary instance.


On the other instance the IVP also fails - and it fails to mount the MD device in shared memory:

SYSTEM on XDGLX2 $ sho dev md
%SYSTEM-W-NOSUCHDEV, no such device available
SYSTEM on XDGLX2 $ @sys$test:decram$ivp shared
Begin DECRAM IVP ...
%SYSTEM-F-BADPARAM, bad parameter value
%SYSTEM-F-DRVERR, fatal drive error
SYSTEM on XDGLX2 $ sho dev md

Device Device Error Volume Free Trans Mnt
Name Status Count Label Blocks Count Cnt
$99$MDS9999: (XDGLX2) Mounted 3 (remote mount) 1
SYSTEM on XDGLX2 $ mou/clu mds9999 mds9999
%MOUNT-F-FORMAT, invalid media format
SYSTEM on XDGLX2 $

Here's DECram's view of the discs:

SYSTEM on XDGLX2 $ decram show disk
From SYS$COMMON:[SYSMGR]MDRECOVER.DAT
Disk $41$MDL0 Size 0 Label MDL0
Disk $99$MDS9999 Size 250 Label MDS9999
Disk $42$MDL0 Size 0 Label MDL0
SYSTEM on XDGLX2 $


Both instances have the same view of memory:

2GB local memory, 2.5GB shared memory (adds up to 6.5GB, so that's OK).

Physical memory I believe has no holes as it's all 2GB pairs (1GB per module) until the final pair which is 512MB (256MB per module).


Anyone else tried this / seen anything like it?

Cheers, Colin (http://www.xdelta.co.uk).

[Colin Butcher]

I'm amazed. No-one else has tried this? Has anyone else got it to work? If so, then how?

Cheers, Colin (http://www.xdelta.co.uk).

[John H. Reinhardt]

No Galaxy license, nor a Galaxy supported system.

[Robert Brooks_1]

Interesting issue -- make sure you bring it up at the boot camp in a couple of weeks.

[John Abbott_2]

Hi Colin, ... don't have DECram... there have been/are some issues around shared memory and VMS V8.2 & V8.3, but I'm fairly sure these are only NUMA system related.

Sorry, that all I know.

J.

[Colin Butcher]

John - do you mean that you don't have them for testing, or do you mean this 4100?

It has a Galaxy license (via DSPP) and that's for the correct number of CPUs.

The 4100 is described in the "Galaxy Guide" and I don't see any reference in the V8.2 or V8.3 release notes about Galaxy support for the 4100 being dropped.

The 4100 memory subsystem works a little differently to that in the ES40 and the GS family, so it wouldn't unduly surprise me if things worked on other machines and not on the 4100.

I can move CPUs around and do all the other Galaxy things you'd expect to do. I've not (yet) written test code to create a shared-memory global section. Nor have I yet gone back to V7.3-2 and V8.2 to test DECram devices in shared memory on those versions.

Rob - yes, see you in Nashua.

Cheers, Colin (http://www.xdelta.co.uk).

[John Abbott_2]

Hi Colin,

I think it's worthing checking if your problem goes away under VMS V7.3-2. I wouldn't bother with 8.2 just yet.

I'm also unaware of Galaxy being dropped on a 4100.

John.

[Colin Butcher]

Hmmm. This is worse than I thought. There aren't any errors (that I can see so far), yet I don't seem to have any communication on the EBA (SMLAN) device as well.

SCS is using EWA by preference. Stopping SCS on EWA (thus forcing it to use EBA) seemed like a good idea - but the instance promptly crashed with a voluntary bugcheck.

Enabling TCPIP on EBA seemed like a good idea too -but I can't PING from instance 0 to instance 1.

Removing 0.5GB memory also seemed like a good idea - so it's down to 6GB, made up of 3x 2GB options.

Still no communication over EBA. Still problems with using shared memory.

Licence is OK as far as I can tell.

GCU output attached as a PDF showing the memory layout. It's a bit odd when looking at the Physical Addresses of the memory fragments.

If I have time (unlikely before the bootcamp) then I'll try V7.3-2 and V8.2

It'd be interesting to see what happens on an ES40 or similar box.

Cheers, Colin (http://www.xdelta.co.uk).

[Colin Butcher]

Darn, forgot the attachement. Hopefully attached here.

Cheers, Colin (http://www.xdelta.co.uk).

[EdgarZamora]

WAG ... What is the value of your SMCI_FLAGS? on both instances.

[Colin Butcher]

Hello,

Interesting. All works fine under V7.3-2 and V8.2. Once I have more information then I'll provide an update here.

Cheers, Colin.

[Colin Butcher]

Hello,

The problem is fixed in patch kit VMS83A_GALAXY_V0100 - recently rleased.

Thank you.

Cheers, Colin (http://www.xdelta.co.uk).

[Colin Butcher]

See above.