VMS hung, MA8000 could be accessed

[Song_Charles]

Hi,

one MA8000 with only 1 FC_switch path was support for 6 es40 VMS Cluster.

OS: OpenVMS V7.3-1 with ECOs(PCSI, Update, SYS, and Fabric_SCSI).

MA8000: 2 HSG80 In dual-redundant configuration

HSG80: Software V86S-1, Hardware E12,
Cache:128 megabyte write cache
Mirrored Cache: 128 megabyte write cache

4 units: 2 member mirror unit used for OS and 3 4-Disk raid5 units was used for customer's database.

after 5 day normally running, the system suddenly couldn't access all disks on MA8000.
restart MA8000 and ES40, the cluster was normal . yesterday,the failure was occurred again. Why?

please refer to:
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1016548

B+R
Charles

[Uwe Zessin · Accepted solution]

Charles,
I might me mis-understanding this:
>one MA8000 with only 1 FC_switch path was support for 6 es40 VMS Cluster.
...
> MA8000: 2 HSG80 In dual-redundant configuration

but last time I checked, OpenVMS requires the MA8000 to run in 'Multibus-Failover' mode.

Do the HSG80 controllers show any errors if you connect to them?

For diagnosing the problem, I would connect two terminal emulators to the maintenance ports of both HSG80 controller modules and record the output for later analysis.

[Petr Spisek]

Unexpected freezing fibre connection could be due wrong configuration of SAN ... many host-ports and few disk-ports.
How looks your configuration of SAN? 6 servers (each with 1 FC adapter) and only one FC path from switch to HSG80?
Petr

[Song_Charles]

yesï¼

just a DSGGB (8 PORTS) FC_SWITCH in Cluster,

all 6 HBA on each ES40 plus 2 port on each HSG80 controllor are centred at FC_switch.

is it wrong configuration?

Charles

[Petr Spisek]

May not be. It's only advice for the way of troubleshooting. I somewhere read an article about this problem, where a lot of companies have a problem with unexpected freezing of SAN. Theory of this is just in configurations with small counter of disk-ports and high number of host-ports.
Sorry, I can't find this article now.
Petr

[Song_Charles]

It's so a pity, that could's the article descripted as my problem.

by the way,
when freezing, 8 green LEDs on FC Switch were still ON, from HSG80, we could find all 6 connecttions (6 HBA's) OFFline status.

Charles

[Vladimir Fabecic]

Hello Charles
First I would like to see output of:
$ show device/multi
from VMS, and
>>> wwidmgr -show wwid -full
from console.
And your FC configuration is incompleted. You have FC switch with 8 ports, and you need 10 ports (6 for HBAs and 4 for HSGs). OK, 8 may be sufficient, but you must be very carefull about which HSG port you connect to FC switch.
And MA8000 must be set in dual-redundant configuration for VMS. I have two VMS clusters and one TRU64 cluster working OK with MA8000, all in dual-redundant configuration.
It would be perfect if you had two FC switches.
Can you send a file which describe your SAN configuration (or drawing)?
Regards

[Vladimir Fabecic]

Oh, sorry
I made a mistake in my previous post.
I wanted to say that Multibus-Failover has to be set and not dual-redundant configuration. And my clusters work with MA8000 with Multibus-Failover mode. (Dahm copy paste).
Also had document that say Multibus-Failover mode is needed for VMS like Uwe said.

[Song_Charles]

Fabecicï¼

Only one HBA(KZPGA-CA)on each ES40 was connected to FC_switch(DSGGB-AA,8 ports).

HSG80 in MA8000 were configurated in dual-redundant.

the right port on bottom HSG80 wasn't function, so I connect the lift port to FC switch, and right port on top HSG80 was connected to FC switch, Is it right?
in another word, only one port on each HSG80 were attach to DSGGB.

Yesterday, I had upgraded the ASC on each HSG80 from V8.6s-1 to V8.6s-13, and if the problem wasn't fixed, I will replace the FC_switch using DSGGC.

give me some suggestion
Thanks

B+R
Charles

[Uwe Zessin]

Charles,
please capture the output of "show this_controller" and attach it as a .TXT file to your next responese. It is important to understand whether your system is running in multibus-failover mode or not. If it is not, you CANNOT simply move to a different controller port, because units 0..99 are tied to Port_1.

[Song_Charles]

HSG80>SHOW THIS|OTHER (message)
------------
HSG> show this

Controller:
HSG80 ZG03713227 Software V86S-1, Hardware E12
NODE_ID = 5000-1FE1-0008-1400
ALLOCATION_CLASS = 1
SCSI_VERSION = SCSI-3
Configured for dual-redundancy with ZG02601287
In dual-redundant configuration
Device Port SCSI address 7
Time: 06-APR-2006 05:16:21
Command Console LUN is lun 0 (IDENTIFIER = 4)
Host PORT_1:
Reported PORT_ID = 5000-1FE1-0008-1401
PORT_1_TOPOLOGY = FABRIC (fabric up)
Address = 011000
Host PORT_2:
Reported PORT_ID = 5000-1FE1-0008-1402
PORT_2_TOPOLOGY = FABRIC (standby)
NOREMOTE_COPY
Cache:
128 megabyte write cache, version 0022
Cache is GOOD
Unflushed data in cache
CACHE_FLUSH_TIMER = DEFAULT (10 seconds)
Mirrored Cache:
128 megabyte write cache, version 0022
Cache is GOOD
Unflushed data in cache
Battery:
NOUPS
FULLY CHARGED
Expires: 21-MAR-2007
HSG>

HSG> show other

Controller:
HSG80 ZG02601287 Software V86S-1, Hardware E12
NODE_ID = 5000-1FE1-0008-1400
ALLOCATION_CLASS = 1
SCSI_VERSION = SCSI-3
Configured for dual-redundancy with ZG03713227
In dual-redundant configuration
Device Port SCSI address 6
Time: 06-APR-2006 05:16:33
Command Console LUN is lun 0 (IDENTIFIER = 4)
Host PORT_1:
Reported PORT_ID = 5000-1FE1-0008-1401
PORT_1_TOPOLOGY = FABRIC (standby)
Host PORT_2:
Reported PORT_ID = 5000-1FE1-0008-1402
PORT_2_TOPOLOGY = FABRIC (fabric up)
Address = 011100
NOREMOTE_COPY
Cache:
128 megabyte write cache, version 0022
Cache is GOOD
No unflushed data in cache
CACHE_FLUSH_TIMER = DEFAULT (10 seconds)
Mirrored Cache:
128 megabyte write cache, version 0022
Cache is GOOD
No unflushed data in cache
Battery:
NOUPS
FULLY CHARGED
Expires: 21-MAR-2007
___________

B+G
Charles

[Uwe Zessin]

Yes, that's what I suspected. You have an invalid configuration (remember: OpenVMS requires multibus-failover) and you cannot move cables, because all units from 0 to 99 are fixed to Port_1 and units 100..199 are fixed to Port_2 in this setup.

[Song_Charles]

Uweï¼

We just one FC switch DSGGB-AA (8 ports), and one of two FCports on Bottom HSG80 had been bad. There are only one HBA on each ES40, in this case, how could I configuate to multibus-failover?

B+G, Charles

[Uwe Zessin]

How do you know it is 'bad'? In dual-redundancy mode, top-controller/Port_2 and bottom-controller/Port_1 are in standby mode and don't show a link. It requires a controller failover to activate a standby port and a bad cable cannot automatically be worked around.

[Song_Charles]

Uwe,

when I connect the fabric cable on that port, the port LED on the FC switch wasn't light, and I just only one FC switch onhand,
How could I make the multibus-failover configurationï¼

and
OpenVMS must requires multibus-failover mode?

B+G
Charles

[Song_Charles]

After UPGRADE ACS FROM V8.6S-1 TO V8,6S-13,
THE CLUSTER HAVEN'T HUNG ABOUT 2 WEEKS, I THINK THE PROBLEMS HAD BEEN SOLVED.


THANKS!
CHARLES