Can't pvcreate 8th external disk

[ED SHERMAN]

HP-UX 11.00 on C240 workstation

I have an external array of 8 hard drives (SCSI IDs 1 through 5 and 8 thru 10) The system disk has SCSI ID 6 and there is some sort of "Initiator" at SCSI ID 7.

I'm trying to put each of the 8 external hard drives as its own volume group (8 volume groups each consisting of 1 disk) and create a single logical volume on each volume group. I can do this successfully for 7 of the 8 external drives but the 8th one hangs when I try to pvcreate /dev/rdsk/c0t10d0.

If I remove one of the other external disks from the SCSI chain then I can pvcreate this hard drive, vgcreate, etc. so I think the disk is okay.

Seems I can only have 7 external drives + 1 internal system disk + 1 "Initiator" thingy on the bus.

Can anyone tell me why I cannot pvcreate that 8th external hard drive?

Thanks,
Ed

Here's my ioscan -fnC disk

# ioscan -fnC disk
Class I H/W Path Driver S/W State H/W Type Description
=============================================================================
disk 6 8/0/19/0.1.0 sdisk CLAIMED DEVICE IBM DDRS-39130WS
/dev/dsk/c0t1d0 /dev/rdsk/c0t1d0
disk 2 8/0/19/0.3.0 sdisk CLAIMED DEVICE IBM DDRS-39130WS
/dev/dsk/c0t3d0 /dev/rdsk/c0t3d0
disk 7 8/0/19/0.4.0 sdisk CLAIMED DEVICE IBM DDRS-39130WS
/dev/dsk/c0t4d0 /dev/rdsk/c0t4d0
disk 8 8/0/19/0.5.0 sdisk CLAIMED DEVICE IBM DDRS-39130WS
/dev/dsk/c0t5d0 /dev/rdsk/c0t5d0
disk 0 8/0/19/0.6.0 sdisk CLAIMED DEVICE SEAGATE ST34572WS
/dev/dsk/c0t6d0 /dev/rdsk/c0t6d0
disk 9 8/0/19/0.8.0 sdisk CLAIMED DEVICE IBM DDRS-39130WS
/dev/dsk/c0t8d0 /dev/rdsk/c0t8d0
disk 10 8/0/19/0.9.0 sdisk CLAIMED DEVICE IBM DDRS-39130WS
/dev/dsk/c0t9d0 /dev/rdsk/c0t9d0
disk 4 8/0/19/0.10.0 sdisk CLAIMED DEVICE IBM DDRS-39130WS
/dev/dsk/c0t10d0 /dev/rdsk/c0t10d0
disk 1 8/16/5.2.0 sdisk CLAIMED DEVICE TOSHIBA CD-ROM XM-6201TA
/dev/dsk/c1t2d0 /dev/rdsk/c1t2d0


Now here's part of a simple ioscan showing that "Initiator" at SCSI ID 7:

# ioscan
H/W Path Class Description
====================================================
bc
8 bc I/O Adapter
8/0 ba GSCtoPCI Bridge
8/0/19/0 ext_bus SCSI C875 Ultra Wide Single-Ended
8/0/19/0.1 target
8/0/19/0.1.0 disk IBM DDRS-39130WS
8/0/19/0.3 target
8/0/19/0.3.0 disk IBM DDRS-39130WS
8/0/19/0.4 target
8/0/19/0.4.0 disk IBM DDRS-39130WS
8/0/19/0.5 target
8/0/19/0.5.0 disk IBM DDRS-39130WS
8/0/19/0.6 target
8/0/19/0.6.0 disk SEAGATE ST34572WS
8/0/19/0.7 target
8/0/19/0.7.0 ctl Initiator
8/0/19/0.8 target
8/0/19/0.8.0 disk IBM DDRS-39130WS
8/0/19/0.9 target
8/0/19/0.9.0 disk IBM DDRS-39130WS
8/0/19/0.10 target
8/0/19/0.10.0 disk IBM DDRS-39130WS

Any help appreciated

Thanks,
Ed

[Helen French]

I would check the following:

1) System patch level. If not updated, install the latest patches.

2) Check the disks and controller with STM utilities. Perform read and write test on the disks with all 8 disks present. This way you can troubleshoot between hardware and OS issues.

3) Check disks with diskinfo, dd commands.

just some thoughts ..

[Dave Chamberlin]

Have you made sure that all device files are present? ie insf -e ?
Also what type of SCSI bus do you have?

[PIYUSH D. PATEL]

Hi,

You can try the following things:

1. Move the disk from the ID 10 to ID 2 which is free and then try.

The last device in a device chain must be terminated. From ID 0-7, 7 ID acts as terminator. The hardware terminators are always there ( sometimes the terminator is inbuilt on the card )

It is also possible that the harddisk at ID 10 may be bad.

#diskinfo /dev/rdsk/c0t10d0

run some disk utilities like dd and check. Or else use stm to check the hardware.

Piyush

[Martin Burnett_2]

Hello Ed,

SCSI ID 7 is always reserved for the controller cards use. This is normal.

Can you access the disk using tar or dd?

# dd if=/dev/rdsk/c0t10d0 bs=64k of=/dev/null

If this command errors with read I/O then you have a bad disk.

Thanks for participating in the forums,

Martin
Chaos reigns within. Reflect, repent, and reboot. Order shall return.

[Sandip Ghosh]

Check your SCSI cables. It seems that you have not connected the proper cable between the controller and the Disk Enclosure.

Sandip

[ED SHERMAN]

Okay,

Sorry about posting an ioscan with SCSI ID 2 free.

I disconnected that disk from the chain so I could pvcreate the disk with SCSI ID 10. I should have mentioned that I think I can have _ANY_ 7 external disks each as their own volume group but cannot pvcreate the remaining 8th disk, whichever one is left over. Didn't try all combinations of course but I did it with SCSI ID 1 pvremoved and got all the remaining disks to work and couldn't pvcreate the disk with SCSI ID 1. Then tried removing disk with SCSI ID 2 from the chain and got the seven attached disks to work but cannot reconnect the disk with SCSI ID 2 and get it to work along with the other 7 disks.

I'm sure the hard drives are fine since I can vi a file and save it to disk and then open the file and read it with vi and delete it with rm. Tried this earlier with each disk when I had that particular disk working.

Anyway here is diskinfo and dd for those who suggested it:

# diskinfo /dev/rdsk/c0t10d0
SCSI describe of /dev/rdsk/c0t10d0:
vendor: IBM
product id: DDRS-39130WS
type: direct access
size: 8886762 Kbytes
bytes per sector: 512

# dd if=/dev/rdsk/c0t10d0 bs=64k of=/dev/null
138855+1 records in
138855+1 records out

I did insf -e and will try again after writing this message, since new device files are reinstalled.

Thanks for info about SCSI ID 7 being reserved for controller cards.

I am curious about Sandip's response -- "Check SCSI Cables... Not using proper cable".

The cable has this label on it:

A1658-62020
REV .D
MDM 3016
M/D 96/33

This is connected to ULTRA WIDE-SE SCSI connector on the back of the C240 Workstation (68 Pin Connector).

I am just now remembering that the vendor who sold me the disks and SCSI cable originally sent me "WIDE DIFFERENTIAL" disks that are not compatible with the ULTRA WIDE-SE SCSI port and I had to send those disks back and get them replaced with the ultra wide single ended disks that I'm using now.

I'll research the cable to see if there is something fishy here.

Also, I suppose I might benefit from finding newer LVM patches.


I'm still working on it.

I'll let you know if I can get this solved.

Thanks,
Ed

[Thayanidhi]

Hi ED,

Please consider the following.

The maximum number of VGs are limited by max_Vg -parameter of kernel. The default value is 10. Just verify whether you crossed this limit?

sysdef - may give you current value.
Either
cd /dev and ll vg*
or
ll /dev/vgxx/group and minor number shoud be like 0x090000 or less.

Make sure no device using SE SCSI mode (its generally a jumper setting. I don't know whether it applies to your case)

--TT

[Nick Wickens · Accepted solution]

The adapter card is SE (single ended) You are limited to 7 devices on SE adapters. If you want more you need a FWD adapter card.

[ED SHERMAN]

Thanks Nick,

I think you have the solution.
I searched the internet to verify your solution and found this table at http://www.storcase.com/tools/downwd.asp

+------------+-------------+--------+--------------------------+--------------+
| SCSI Type | Bus Speed | Bus | Max Bus Length (meters) | Max. Devices |
| | max. MB/sec | Width +----------+-------+-------+ Supported |
| | | | S.E. | LVD | HVD | |
+------------+-------------+--------+----------+-------+-------+--------------+
| Narrow | 5 | 8 | 6 | - | 25 | 8 |
| Fast | 10 | 8 | 3 | - | 25 | 8 |
| Fast Wide | 20 | 16 | 3 | - | 25 | 16 |
| Ultra | 20 | 8 | 1.5 | - | 25 | 8 |
| Ultra | 20 | 8 | 3 | - | - | 4 |
| Wide Ultra | 40 | 16 | - | - | 25 | 16 |
| Wide Ultra | 40 | 16 | 1.5 | - | - | 8 |
| Wide Ultra | 40 | 16 | 3 | - | - | 4 |
| Ultra2 | 40 | 8 | - | 12 | 25 | 8 |
| Wide Ultra2| 80 | 16 | - | 12 | 25 | 16 |
| Ultra160 | 160 | 16 | - | 12 | - | 16 |
| Ultra320 | 320 | 16 | - | 12 | - | 16 |
+------------+-------------+--------+----------+-------+-------+--------------+


(I hope that table doesn't wrap when I post it)

The table says Ultra Wide SE with 1.5 meter cable can support 8 devices and only 4 devices if I use a 3 meter cable. I'm using a 2.5 meter cable so I guess I'm doing pretty good!

Ultra Wide HVD can support 16 devices and Fast Wide SE can support 16 devices.

Thanks everyone for your posts!
You guys really helped me out with suggestions that I didn't know to try.

[Thayanidhi]

Hi nick,

If you see output of ioscan it shows ultra wide.

More over if it is single ended there is no scsi id more than 7. In this case the system is able see the devices beyond id 7!!

I don't think it is the issue.

--TT

[Thayanidhi]

Hi ED,

As you suspected the table got wraped!!

Any way I could manipulate.

[Nick Wickens]

Hi

In case I had given incorrect information I did a quick search of the net and this link confirms that although the bus is Ultra Wide, which would give it a 16bit bus width it is unfortunatly still Single Ended and therefore it can still only support 8 devices. There is a table off of the link.

http://www.pcguide.com/ref/hdd/if/scsi/confNumber-c.html

[Thayanidhi]

Hi Nick,

You are correct, I got confused between SE and wide SCSI.

In general all wide SCSI support 16 devices,
I didn't know that it is limited cable length too!

[Angus Crome]

Just for quick clarification in the future;

Single Ended and Differential are only pertinent for cable distances.

Narrow and Wide refer to the bit width of the cable bus (8 for narrow, 16 for wide). This give you an intrinsic definition of why there are 8 or 16 devices allowed for each bus. Although, you should never really use any more than 4 or 6 devices on bus, but we all do anyway.

SCSI, ,SCSI2, Ultra SCSI, Ultra2 SCSI, Ultra3 SCSI, Ultra-160 and Ultra-320 define the transfer speed of the bus.

Note: I get shaky here, because the definitions shifted between SCSI-2 and Ultra.

SCSI 5Mb/s
SCSI2 10Mb/s
Ultra SCSI 20Mb/s
### These two may be off
Ultra2 SCSI 40Mb/s
Ultra3 SCSI 80Mb/s
### End
Ultra-160 SCSI 160Mb/s
Ultra-320 SCSI 320Mb/s

HVD (High Voltage Differential) SCSI is 20 Mb/s and has not been pushed forward.

LVD (Low Voltage Differential) SCSI is a method for switching between speeds to match the speed of the controller. The differential in this name has nothing to do with cable distance.

[Eric Burns_1]

Okay, to clear up some people's statements about scsi IDs and transfer mode, Indeed, they have nothing to do with each other, but rather a wide bus allows for 16 devices, and only the original scsi limit of 8 devices on a narrow scsi bus. Generally low voltage differential drives can automatically drop down to single-ended mode if they have to because other devices on the bus are single ended. However the term "differential" generally refers to the older, incompatible high voltage differential. So ensuring that all devices are in single-ended mode is not a problem. And someone thinking there is a limit of 7 scsi devices also does not make sense since an ID of 8 9 or 10 could not work in such a case and would conflict with devices 0, 1, and 2 respectively. But again, that is not a problem. The statement that LVD does not have anything to do with cable length, is also wrong. Differential's (both high and low's) reason for introduction was to overcome cable length limitations by replacing ground signals with reverse matching reverse polatity data signals so that interference could be recognized and accounted for. The table you listed before saying that you should have a limit of 1.5 meters for 8 devices on a single ended ultra-wide connection is what you should probably worry about if you have hanging problems (normally caused by bus errors) occuring. You saying you have 2.5 meters means you're not doing good.. you're way over the length. Also, note that the total length is not only for the external devices you have, but also the length of the cable inside the computer as well. You might do well with a scsi enclosure that can house all 8 of your IBM drives in one case with limited cable length.