MSA Storage Disks Upgrade

[rianui]

Hi Gurus,

I have this disk enclosule with seven disks populated and it connects to an rp3440 server (UX 11.11) through two SCSI cables at the back. I recently added two more SCSI disks to increase space to an existing volume. However when I ran ioscan i do not see the two new drives but only the older drives. On the front of the enclosure I can see "hp storageworks" written but I cannot really figure out what storage it is. It has two blowers at the back, same as those on an EVA 4000 disk enclosure. I concluded it is an MSA 1000 or 1500. I downloaded an Array Configuration Utility software and installed it on the server but it seems somewhat complicated to use. Is this the best way to find my new drives or is there a better way to go about this?

Regards,

Rianui

[Torsten.]

There are so many MSA's, like MSA30, MSA1000, MSA1500, etc ...


Could you please post your


# ioscan -fn


to figure out what it really is?

[rianui]

Hi Torsten,

Find attached.

Regards,

Rianui

[Torsten. · Accepted solution]

Look at this:

ext_bus     4  0/4/1/0/4/0    ciss      CLAIMED     INTERFACE    PCI-X SmartArray 6404 RAID Controller
                             /dev/ciss4
target      7  0/4/1/0/4/0.0  tgt       CLAIMED     DEVICE
disk        3  0/4/1/0/4/0.0.0  sdisk     CLAIMED     DEVICE       HP      LOGICAL VOLUME
                             /dev/dsk/c4t0d0   /dev/rdsk/c4t0d0
target      8  0/4/1/0/4/0.7  tgt       CLAIMED     DEVICE
ctl         4  0/4/1/0/4/0.7.0  sctl      CLAIMED     DEVICE       Initiator
                             /dev/rscsi/c4t7d0
ext_bus     5  0/4/1/0/5/0    ciss      CLAIMED     INTERFACE    PCI-X SmartArray 6404 RAID Controller
                             /dev/ciss5
target      9  0/4/1/0/5/0.0  tgt       CLAIMED     DEVICE
disk        4  0/4/1/0/5/0.0.0  sdisk     CLAIMED     DEVICE       HP      LOGICAL VOLUME
                             /dev/dsk/c5t0d0   /dev/rdsk/c5t0d0
target     10  0/4/1/0/5/0.7  tgt       CLAIMED     DEVICE

Your enclosure is most likely a MSA30 connected to a smart array sa6400.

You probably know this from Proliant.

You need to create or extend the logical drives, the physical disks are not visible from the OS.

To get more information do

# sautil /dev/ciss4

# saconfig /dev/ciss4

# sautil /dev/ciss5

# saconfig /dev/ciss5

Since you cannot extend a physical volume in HP-UX 11.11, you should consider to create a new logical drive and extend the volume group in question.

I can guzide you, if you send the information above.

[rianui]

Hi Torsten,

Thanks. Find attached.

Just some background info about the system. When I first checked the server, I realized that one of the two SCSI cables to the enclosure had been disconnected. ioscan could only show fewer than the disks in the enclosure. But since the production databse was running no one noticed any problem so I guess the unavailed disks accessed through SCSI cable 2 did not have any serious application on them.

Regards,

Rianui

[Torsten.]

Well, you loose the mirrors of the logical drive (disconnected cable).


If a disk fails now, you loose your data !!!!


You should (if possible) connect the cable and reboot the server.


If no reboot possible, connect the cable and run the commands again. Send the status!


Note: one of your logical drives is already FAILED!

[Torsten.]

This is a very good example for very bad administration.

Let's see the details:

- all 4 cache batteries are failed, so you don't have write cache (slow writes)

- on ciss4 there are 3 missing disks in the RAID, so it isn't fully mirrored

- the RAID at ciss5 is completely FAILED, because 7 disks are missing

Your 2 new disks are detected, but you should first fix all the problems!!!

[rianui]

Hi Torsten,

- This is a very good example for very bad administration.

A. Totally agree! There is no inhouse admin and those on site avoid the system.....as long as the business runs.

- Well, you loose the mirrors of the logical drive (disconnected cable).

A. The SCSI cables are both connected but the client wouldn't allow that before end of working hours.

- You should (if possible) connect the cable and reboot the server.

A. Connected.

- If no reboot possible, connect the cable and run the commands again. Send the status!

A. Please find attached.

- Note: one of your logical drives is already FAILED!

A. I cannot physically identify the disk. After running the "hpacucli" command yesterday the disks went red (which I find happens all the time I run an array utility) but even after exiting the utility the disks are still displaying the red alert as though the utility is still on.

- all 4 cache batteries are failed, so you don't have write cache (slow writes)

A. May I ask where t6he batteries would be? The enclosure connects directly to the server and the only visible components are at the back (two blowers). It does not have any controllers. Correct me if these (batteries) are features within the enclosure.

- on ciss4 there are 3 missing disks in the RAID, so it isn't fully mirrored

- the RAID at ciss5 is completely FAILED, because 7 disks are missing

- Your 2 new disks are detected, but you should first fix all the problems!!!

A. As I have mentioned, the cables are connected. I have checked them again. Also, ALL disks are displaying the red light.

Regards,

Rianui

[Torsten.]

The batteries are on the smartarray controller HBA inside the server!

I will check your attachments.

[Torsten.]

All logs look like before.

Try again after a server reboot.


Regarding the tool:

hpacucli is indented to manage the MSA1000/1500, not this controller, but obviously it speaks to the controller.

Let's try to show something:

hpacucli> ctrl all show config

hpacucli> ctrl all show config detail



Here is an example for switching off the LEDs:

hpacucli> ctrl slot=0 ld 0 modify led=off

(not sure what is the correct value for "slot" here, hence the "show" commands)

[Torsten.]

Meanwhile you can try

# sautil /dev/ciss5 scan
(if your version has this option)

or

# sautil /dev/ciss5 reset_adapter

Both will force the controller to search for the disks, after you reconnected the cable.

Generate the sautil output again after this.

[rianui]

Hi Torsten,

I have gotten a windows and restarted both the server and the disk enclosure.

# sautil /dev/ciss5 scan

******************************************************************************
****                                                                      ****
****             S A U T I L   S u p p o r t   U t i l i t y              ****
****                                                                      ****
****             for the HP SmartArray RAID Controller Family             ****
****                                                                      ****
****                           version A.02.06                            ****
****                                                                      ****
****  (C) Copyright 2003, 2004 Hewlett-Packard Development Company, L.P.  ****
******************************************************************************

---- DRIVER INFORMATION ------------------------------------------------------

  Driver State........................ READY

---- CONTROLLER INFORMATION --------------------------------------------------

  Controller Product Number........... A9891A
  Controller Product Name............. HP PCI-X SmartArray 6404 (Ch B1, B2)
  Hardware Path....................... 0/4/1/0/5/0
  Device File......................... /dev/ciss5

---- SCAN SCSI BUSES ---------------------------------------------------------

  Scanning all SCSI buses (this could take a few minutes).... [Done]

******************************************************************************
****                         End of SAUTIL Output                         ****
******************************************************************************

See attached the output of the commands again.

Regards,

Rianui

[Torsten.]

This is what we have now:

Controller Product Name............. HP PCI-X SmartArray 6404 (Ch A1, A2)
  Hardware Path....................... 0/4/1/0/4/0
  Serial Number....................... P57820P9SRU0LD
  Device File......................... /dev/ciss4

---- SCSI DEVICE SUMMARY -----------------------------------------------------

  Location   Ch    ID    Type         Capacity   Status

  N/A         1     0    N/A         N/A         FAILED
  N/A         1     1    N/A         N/A         FAILED
  N/A         1     2    N/A         N/A         FAILED
  external    2     0    DISK         36.4 GB    OK
  external    2     1    DISK         36.4 GB    OK
  external    2     2    DISK         36.4 GB    OK
  external    2     3    DISK         36.4 GB    OK
  external    2     4    DISK         36.4 GB    OK
  external    2     5    DISK         36.4 GB    OK
  external    2     8    DISK         36.4 GB    OK
  external    2    15    PROCESSOR

Controller Product Name............. HP PCI-X SmartArray 6404 (Ch B1, B2)
  Hardware Path....................... 0/4/1/0/5/0
  Serial Number....................... P577C0M9SRT05Q
  Device File......................... /dev/ciss5

---- SCSI DEVICE SUMMARY -----------------------------------------------------

  Location   Ch    ID    Type         Capacity   Status

  external    1     0    DISK         36.4 GB    OK
  external    1     1    DISK         36.4 GB    OK
  external    1     2    DISK         36.4 GB    OK
  external    1     3    DISK         36.4 GB    UNASSIGNED
  external    1     4    DISK         36.4 GB    UNASSIGNED
  external    1     7    PROCESSOR
  N/A         2     0    N/A         N/A         FAILED
  N/A         2     1    N/A         N/A         FAILED
  N/A         2     2    N/A         N/A         FAILED
  N/A         2     3    N/A         N/A         FAILED
  N/A         2     4    N/A         N/A         FAILED
  N/A         2     5    N/A         N/A         FAILED
  N/A         2     8    N/A         N/A         FAILED

Can you please tell about the cable connections between server and MSA (e.g. CH A1 to left half of MSA, etc)

and

which slots of the MSA are now populated?

           ========---========
          |                   |
          |       U320    <--- LOOK FOR THIS LOGO
          |        / \        |
          |         <-        |
          |        \ /        |
          |                   |
          |     *       *     |
          |   * *       * *   |
  (Ch A1)---> * *       * * <--- (Ch A2)
          |   * *       * *   |
          |     *       *     |
          |                   |
          |     *       *     |
          |   * *       * *   |
  (Ch B1)---> * *       * * <--- (Ch B2)

          |   * *       * *   |
          |     *       *     |
          |                   |
          |                   |
           ~   ~   ~   ~   ~

[Torsten.]

I'm sure the cables are wrong connected now.

[Torsten.]

The SA6404 is in fact a two in one controller.

This means there is a single card with 2 SA6402 (independent) controllers on it.


The configuration was done with all disks connected to a single controller (either A1/A2 or B1/B2), but now the half is connected to A, the other half to B.


We have to bring them together again.

[Torsten.]

Can you please post

# strings /etc/lvmtab

and

# vgdisplay -v

[rianui]

Find attached.

Yes, the server has one SCSI card with four slots. Two slots are parallel to the other two. The connections are done this way:

- Cable one is connects to the upper right slot

- Cable two is connected to lower left slot.

- Two slots are unused

Regards,

Rianui

[Torsten.]

I'm sure both cables must be connect to either both A or both B.


Question is A or B?

The external disks are not under LVM control - how are they used?

[Torsten.]

Which cables was disconnected?

[rianui]

The cable on the left. I have connected it to the port next to "cable 1" and restarted.

Let me send you the output in lvmtab in a little while.

[rianui]

So, I have moved one cable to both slots that were unused before and restarted both times. I then captured lvmtab info each time. I have also noted that ioscan this two times shows one LOGICAL VOLUME under Description when I moved the cable from it's original position.

# ioscan -fnCdisk
Class     I  H/W Path       Driver   S/W State   H/W Type     Description
==========================================================================
disk      0  0/0/2/0.0.0.0  sdisk    CLAIMED     DEVICE       TEAC    DV-28E-C
                           /dev/dsk/c0t0d0   /dev/rdsk/c0t0d0
disk      1  0/1/1/0.0.0    sdisk    CLAIMED     DEVICE       HP 36.4GST336754LC
                           /dev/dsk/c2t0d0   /dev/rdsk/c2t0d0
disk      2  0/1/1/0.1.0    sdisk    CLAIMED     DEVICE       HP 36.4GST336754LC
                           /dev/dsk/c2t1d0   /dev/rdsk/c2t1d0
disk      3  0/4/1/0/4/0.0.0  sdisk    CLAIMED     DEVICE       HP      LOGICAL VOLUME
                           /dev/dsk/c4t0d0   /dev/rdsk/c4t0d0

When I take "cable 2" back to it's original position, ioscan gives the output below.

# ioscan -fnCdisk
Class     I  H/W Path       Driver   S/W State   H/W Type     Description
==========================================================================
disk      0  0/0/2/0.0.0.0  sdisk    SCAN        DEVICE       TEAC    DV-28E-C
                           /dev/dsk/c0t0d0   /dev/rdsk/c0t0d0
disk      1  0/1/1/0.0.0    sdisk    SCAN        DEVICE       HP 36.4GST336754LC
                           /dev/dsk/c2t0d0   /dev/rdsk/c2t0d0
disk      2  0/1/1/0.1.0    sdisk    SCAN        DEVICE       HP 36.4GST336754LC
                           /dev/dsk/c2t1d0   /dev/rdsk/c2t1d0
disk      3  0/4/1/0/4/0.0.0  sdisk    SCAN        DEVICE       HP      LOGICAL VOLUME
                           /dev/dsk/c4t0d0   /dev/rdsk/c4t0d0
disk      4  0/4/1/0/5/0.0.0  sdisk    SCAN        DEVICE       HP      LOGICAL VOLUME
                           /dev/dsk/c5t0d0   /dev/rdsk/c5t0d0

See the results attached.

Regards,

Rianui

[Torsten.]

Please end the sautil output again.

[rianui]

Hi Torsten,

The client has closed for the day. Let me send it tomorrow.

Also, I have returned the cable to the original slot. Do you want me to run the command after changing the cables to the other slot?

Regards,

Rianui

[Torsten.]

yes.

Do the sautil scan command before get the sautil output.

[Torsten.]

Short summary:

At the beginning both cables were connected to a single HBA (the "A" connectors). Then one of the cables was removed.

So the HBA looses some disks of the mirrored RAID set.

While connecting the cable now to a "B" port, the second (independend) controller detects the disks as *another* RAID, but with too less disks and marked this "second" RAID as failed.


If you now connect the cable back to the "A" port, the controller will detect the disks as part of the existing RAID and merge the RAID together.

Once we have the status, we can decide how to proceed with the 2 new disks.


I still wonder how the RAID is used.


It is not under LVM control.

Maybe used as a plain disk?

# mount

may tell.

Maybe VxVM?

# vxdisk list

may tell.


The problem is:

The controller is able to extend the existing RAID, but the OS most likely cannot handle the growing "disk".