LSM - Recovering from disks being NODEVICE ??

[Gary Hansford]

We have an EVA storage system in site 'A(MIS)' and 'B(A58)' along with an ES45 5.1b machine. We use LSM to mirror the system disks and data between the sites 'A' and 'B'.

Yesterday, we lost site 'A' along with quorum, had to manual boot site 'B'. LSM Mirror'd system disk came up but no other LSM volumes were present... Did a "voldg -f import " and "volume -g startall" to present the missing volumnes and enable site 'B' system to work...

Today site 'A' is back, it was basically a power outage. I can see all the disks and such like, but how can I get LSM to reuse these drives ?? (I don't want to reboot site 'A's alpha until I am sure site 'B' has LSM back to normal !!).

Here is output from commands on site 'B': -

# hwmgr -view dev -cat disk
HWID: Device Name Mfg Model Location
------------------------------------------------------------------------------
85: /dev/disk/dsk9c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=3
86: /dev/disk/dsk10c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=4
87: /dev/disk/dsk11c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=6
90: /dev/disk/dsk12c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=10
91: /dev/disk/dsk13c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=11
92: /dev/disk/dsk14c COMPAQ HSV110 (C)COMPAQ bus-5-targ-2-lun-6
93: /dev/disk/dsk15c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=108
94: /dev/disk/dsk16c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=106
95: /dev/disk/dsk17c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=110
96: /dev/disk/dsk18c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=111
97: /dev/disk/dsk19c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=8
98: /dev/disk/dsk20c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=1
99: /dev/disk/dsk21c COMPAQ HSV110 (C)COMPAQ IDENTIFIER=102
184: /dev/disk/dsk22c COMPAQ BD0186459A bus-0-targ-0-lun-0
185: /dev/disk/dsk23c COMPAQ BD0186459A bus-1-targ-0-lun-0
313: /dev/disk/floppy2c 3.5in floppy fdi0-unit-0
361: /dev/disk/cdrom3c HL-DT-ST DVD-ROM GDR8161B bus-3-targ-0-lun-0

# voldctl enable

# voldisk list
DEVICE TYPE DISK GROUP STATUS
dsk9 sliced - - unknown
dsk10 sliced - - unknown
dsk11 sliced - - unknown
dsk12 sliced - - online
dsk13 sliced - - online
dsk14 sliced - - unknown
dsk15 sliced a58_dsk15 rootdg online
dsk16 sliced - - unknown
dsk17 sliced a58_dsk17 batchdg online
dsk18 sliced a58_dsk18 ingresdg online
dsk19 sliced - - online
dsk20 sliced - - unknown
dsk21 sliced - - unknown
dsk22 sliced - - unknown
dsk23 sliced - - unknown
- - mis_dsk19 rootdg failed was:dsk19
- - mis_dsk12 batchdg failed was:dsk12
- - mis_dsk13 ingresdg failed was:dsk13

Best of ALL the VOLPRINT... NODEVICE !!!

# volprint -hAt
Disk group: rootdg

DG NAME NCONFIG NLOG MINORS GROUP-ID
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE

dg rootdg default default 0 1126173100.1026.srukfischcl

dm a58_dsk15 dsk15 sliced 4096 41938928 -
dm mis_dsk19 - - - - NODEVICE

v cluster_rootvol cluroot ENABLED ACTIVE 4276096 SELECT -
pl a58_root_plex1 cluster_rootvol ENABLED ACTIVE 4276096 CONCAT - RW
sd a58_dsk15-01 a58_root_plex1 a58_dsk15 0 4276096 0 dsk15 ENA
pl mis_root_plex1 cluster_rootvol DISABLED NODEVICE 4276096 CONCAT - RW
sd mis_dsk19-01 mis_root_plex1 mis_dsk19 0 4276096 0 - RLOC

v cluster_usrvol fsgen ENABLED ACTIVE 21045856 SELECT -
pl a58_usr_plex1 cluster_usrvol ENABLED ACTIVE 21045856 CONCAT - RW
sd a58_dsk15-02 a58_usr_plex1 a58_dsk15 4276486 21045856 0 dsk15 ENA
pl mis_usr_plex1 cluster_usrvol DISABLED NODEVICE 21045856 CONCAT - WO
sd mis_dsk19-02 mis_usr_plex1 mis_dsk19 4276096 21045856 0 - RLOC
pl a58_usr_plex2 cluster_usrvol ENABLED ACTIVE LOGONLY CONCAT - RW
sd a58_dsk15-03 a58_usr_plex2 a58_dsk15 4276096 390 LOG dsk15 ENA

v cluster_varvol fsgen ENABLED ACTIVE 16615734 SELECT -
pl mis_var_plex1 cluster_varvol DISABLED NODEVICE 16615734 CONCAT - WO
sd mis_dsk19-03 mis_var_plex1 mis_dsk19 25321952 16615734 0 - RLOC
pl a58_var_plex1 cluster_varvol ENABLED ACTIVE 16615734 CONCAT - RW
sd a58_dsk15-04 a58_var_plex1 a58_dsk15 25322667 16615734 0 dsk15 ENA
pl a58_var_plex2 cluster_varvol ENABLED ACTIVE LOGONLY CONCAT - RW
sd a58_dsk15-05 a58_var_plex2 a58_dsk15 25322342 325 LOG dsk15 ENA

Disk group: batchdg

DG NAME NCONFIG NLOG MINORS GROUP-ID
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE

dg batchdg default default 1000 1126177441.1121.srukfischcl

dm a58_dsk17 dsk17 sliced 4096 62910448 -
dm mis_dsk12 - - - - NODEVICE

v batchvol fsgen ENABLED ACTIVE 62908416 SELECT -
pl mis_plex1 batchvol DISABLED NODEVICE 62908416 CONCAT - WO
sd mis_dsk12-01 mis_plex1 mis_dsk12 1040 62908416 0 - RLOC
pl a58_plex1 batchvol ENABLED ACTIVE 62908416 CONCAT - RW
sd a58_dsk17-01 a58_plex1 a58_dsk17 0 62908416 0 dsk17 ENA
pl mis_plex2 batchvol DETACHED STALE LOGONLY CONCAT - WO
sd mis_dsk12-02 mis_plex2 mis_dsk12 0 1040 LOG - NDEV

Disk group: ingresdg

DG NAME NCONFIG NLOG MINORS GROUP-ID
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE

dg ingresdg default default 4000 1126177540.1136.srukfischcl

dm a58_dsk18 dsk18 sliced 4096 629141488 -
dm mis_dsk13 - - - - NODEVICE

v ingresvol fsgen ENABLED ACTIVE 629135360 SELECT -
pl mis_plex1 ingresvol DISABLED NODEVICE 629135360 CONCAT - WO
sd mis_dsk13-01 mis_plex1 mis_dsk13 4160 629135360 0 - RLOC
pl a58_plex1 ingresvol ENABLED ACTIVE 629135360 CONCAT - RW
sd a58_dsk18-01 a58_plex1 a58_dsk18 0 629135360 0 dsk18 ENA
pl mis_plex2 ingresvol DETACHED STALE LOGONLY CONCAT - WO
sd mis_dsk13-02 mis_plex2 mis_dsk13 0 4160 LOG - NDEV


How should I go about recovering this. On 4.0f I used to treat it as though the whole drive had gone, delete and replace scenario.. Is there a better way - and what about the system disk mirror'd drive - how should I approach that... ??

Answers gratefully received...

Cheers All

Gary

[Venkatesh BL · Accepted solution]

I checked with some LSM folks around...here is what I got:

+++++++++++++++++++++++++++++++++++
They can simply remove the disk from LSM and add it back to the disk-group with -k option.
Then do a volrecover.
These are the following steps to follow.
===============
voldg -g -k rmdisk
voldisk rm
voldisksetup -i
volg -g -k adddisk
volrecover -sb
================
+++++++++++++++++++++++++++++++++++

[Gary Hansford]

BL was spot on with the answer - had Idris from HP helping us out... It was as always - remove disk and re-add !! Here's how it was done here (using voldiskadm for some of the steps) :-)

1. voldiskadm - option (4) "Remove a disk for replacement"...

2. voldisk rm

3. voldiskadm - option (1) "Add or initialize one or more disks"...

4. voldg -g -k adddisk =

5. volrecover -g -sb

This included the system disk etc...

Cheers Everyone

Gary