couldnt mount a mount point after reboot

[silusan]

Hpux  server

swap 100%  full

commands not working..too slow.. no proper typing...so rebooted the server without noting down any information.

(Our guy rebooted the server after he got permission)

Rebooted properly and swap came down to 12% and then he handed it back to the DBAs

But DBAs came back and said a particular mount point ( /oradata)  is missing.

Now this is the data from the server(after reboot)

 :/ # strings /etc/lvmtab
/dev/vg00
/dev/dsk/c0t0d0s2
 :/ #
 :/ #
 :/ # ioscan -funC disk
Class     I  H/W Path     Driver  S/W State   H/W Type     Description
=======================================================================
disk      0  0/0/0/0.0.0  sdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                         /dev/dsk/c0t0d0     /dev/dsk/c0t0d0s2   /dev/rdsk/c0t0d0    /dev/rdsk/c0t0d0s2
                         /dev/dsk/c0t0d0s1   /dev/dsk/c0t0d0s3   /dev/rdsk/c0t0d0s1  /dev/rdsk/c0t0d0s3
disk      2  0/0/0/0.2.0  sdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                         /dev/dsk/c0t2d0   /dev/rdsk/c0t2d0
 :/ #

 :/ # cat /etc/fstab
# System /etc/fstab file.  Static information about the file systems
# See fstab(4) and sam(1M) for further details on configuring devices.
/dev/vg00/lvol3 / vxfs delaylog 0 1
/dev/vg00/lvol1 /stand vxfs tranflush 0 1
/dev/vg00/lvol4 /home vxfs delaylog 0 2
/dev/vg00/lvol5 /opt vxfs delaylog 0 2
/dev/vg00/lvol6 /tmp vxfs delaylog 0 2
/dev/vg00/lvol7 /usr vxfs delaylog 0 2
/dev/vg00/lvol8 /var vxfs delaylog 0 2
/dev/vg00/osmf /osmf vxfs delaylog 0 2
/dev/vg00/lvol9 /var/adm/crash vxfs delaylog 0 2
/dev/vg00/lv-oracle /opt/oracle vxfs delaylog 0 3
/dev/dsk/c0t2d0 /oradata vxfs delaylog 0 3
:/#

 :/ #

after reboot :

 :/ # bdf         (oradata mount point missing!)
Filesystem          kbytes    used   avail %used Mounted on
/dev/vg00/lvol3    1048576  298504  744248   29% /
/dev/vg00/lvol1     524288  310160  212480   59% /stand
/dev/vg00/lvol8    6291456 4387808 1894408   70% /var
/dev/vg00/lvol9    3145728  268544 2697424    9% /var/adm/crash
/dev/vg00/lvol7    4194304 2568696 1612968   61% /usr
/dev/vg00/lvol6    4194304 1691512 2483488   41% /tmp
/dev/vg00/osmf     1572864  184581 1301835   12% /osmf
/dev/vg00/lvol5    6291456 3871984 2400624   62% /opt
/dev/vg00/lv-oracle 10485760 10304747  181013   98% /opt/oracle
/dev/vg00/lvol4     524288   22008  498400    4% /home
 :/ #

couldn’t mount/ oradata

 :/oradata # mount -F vxfs /dev/dsk/c0t2d0 /oradata
UX:vxfs mount: ERROR: V-3-20023: file system on /dev/dsk/c0t2d0 has structural damage
 :/oradata #

 :/ # fsck -F vxfs -n -o full /dev/rdsk/c0t2d0
UX:vxfs fsck: ERROR: V-3-20727: OLT extent 0 has bad magic
read of primary OLT failed
UX:vxfs fsck: ERROR: V-3-20727: OLT extent 1 has bad magic
read of OLT copy failed
UX:vxfs fsck: ERROR: V-3-20718: no valid OLT, cannot continue
 :/ #

 :/ # vgdisplay -v vg00
--- Volume groups ---
VG Name                     /dev/vg00
VG Write Access             read/write
VG Status                   available
Max LV                      255
Cur LV                      11
Open LV                     11
Max PV                      8
Cur PV                      1
Act PV                      1
Max PE per PV               9200
VGDA                        2
PE Size (Mbytes)            32
Total PE                    3171
Alloc PE                    1296
Free PE                     1875
Total PVG                   0
Total Spare PVs             0
Total Spare PVs in use      0

   --- Logical volumes ---
   LV Name                     /dev/vg00/lvol1
   LV Status                   available/syncd
   LV Size (Mbytes)            512
   Current LE                  16
   Allocated PE                16
   Used PV                     1

   LV Name                     /dev/vg00/lvol2
   LV Status                   available/syncd
   LV Size (Mbytes)            4096
   Current LE                  128
   Allocated PE                128
   Used PV                     1

   LV Name                     /dev/vg00/lvol3
   LV Status                   available/syncd
   LV Size (Mbytes)            1024
   Current LE                  32
   Allocated PE                32
   Used PV                     1

   LV Name                     /dev/vg00/lvol4
   LV Status                   available/syncd
   LV Size (Mbytes)            512
   Current LE                  16
   Allocated PE                16
   Used PV                     1

   LV Name                     /dev/vg00/lvol5
   LV Status                   available/syncd
   LV Size (Mbytes)            6144
   Current LE                  192
   Allocated PE                192
   Used PV                     1

   LV Name                     /dev/vg00/lvol6
   LV Status                   available/syncd
   LV Size (Mbytes)            4096
   Current LE                  128
   Allocated PE                128
   Used PV                     1

   LV Name                     /dev/vg00/lvol7
   LV Status                   available/syncd
   LV Size (Mbytes)            4096
   Current LE                  128
   Allocated PE                128
   Used PV                     1

   LV Name                     /dev/vg00/lvol8
   LV Status                   available/syncd
   LV Size (Mbytes)            6144
   Current LE                  192
   Allocated PE                192
   Used PV                     1

   LV Name                     /dev/vg00/lvol9
   LV Status                   available/syncd
   LV Size (Mbytes)            3072
   Current LE                  96
   Allocated PE                96
   Used PV                     1

   LV Name                     /dev/vg00/osmf
   LV Status                   available/syncd
   LV Size (Mbytes)            1536
   Current LE                  48
   Allocated PE                48
   Used PV                     1

   LV Name                     /dev/vg00/lv-oracle
   LV Status                   available/syncd
   LV Size (Mbytes)            10240
   Current LE                  320
   Allocated PE                320
   Used PV                     1

   --- Physical volumes ---
   PV Name                     /dev/dsk/c0t0d0s2
   PV Status                   available
   Total PE                    3171
   Free PE                     1875
   Autoswitch                  On
   Proactive Polling           On

[donna hofmeister]

this might be helpful: http://www.symantec.com/business/support/index?page=content&id=TECH69086

but i strongly suggest you open a call with the RC....  

[Turgay Cavdar]

Hi,

>>Rebooted properly
IS this clean reboot? Or hard reset/TOC? Are there any error logs during shutdown and afterwards reboot?

By the way it seems this server is a virtual machine, any possibility that the disk (/dev/dsk/c0t2d0) is toasted accidentally.

>>UX:vxfs fsck: ERROR: V-3-20727: OLT extent 0 has bad magic read of primary OLT failed
It seems your file system is corrupted, you may need to restore from backups...

[silusan]

# cat /etc/fstab
# System /etc/fstab file.  Static information about the file systems
# See fstab(4) and sam(1M) for further details on configuring devices.
/dev/vg00/lvol3 / vxfs delaylog 0 1
/dev/vg00/lvol1 /stand vxfs tranflush 0 1
/dev/vg00/lvol4 /home vxfs delaylog 0 2
/dev/vg00/lvol5 /opt vxfs delaylog 0 2
/dev/vg00/lvol6 /tmp vxfs delaylog 0 2
/dev/vg00/lvol7 /usr vxfs delaylog 0 2
/dev/vg00/lvol8 /var vxfs delaylog 0 2
/dev/vg00/osmf /osmf vxfs delaylog 0 2
/dev/vg00/lvol9 /var/adm/crash vxfs delaylog 0 2
/dev/vg00/lv-oracle /opt/oracle vxfs delaylog 0 3
/dev/dsk/c0t2d0 /oradata vxfs delaylog 0 3
:/#

what could be the reason.... there is no VG name associated with /oradata?

[Turgay Cavdar]

The reason is the disk was not included in any vg, the file system was created directly on the disk device file.

[silusan]

the file system was created directly on the disk device file. please tell me why do we do that ... why not disk..VG..LV and then an FS on top? and how do we do that(FS directly on the disk device file) thanks

[Matti_Kurkela]

You should ask the person who did that, because it is definitely not a good procedure.

Possible reasons would be:

• whoever set up that filesystem was not familiar with LVM and was afraid to use it
• or maybe someone thought LVM causes extra overhead (technically this is true, but the overhead is very very minimal and LVM is much more flexible than FS-on-disk-device: easier capacity management, easier storage migrations even on-line, etc...)