LUN Limit - Question/Explanation

[Tom Dawson]

Hi,

On 8/29, Dieter Vener posted a thread about having a
problem seeing LUNS:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=952270

In a response, Tim Nelson mentioned that a "lun cannot be
over 8 (0-7). Then gives the example:

First luns 0-7
Next luns 10-17
next luns 20-28

So I have three questions I hope someone can answer/clarify
for me:

1. Does HP-UX impose a limit of 0-7 on its' LUNs?
2. Where do the "10-17" and "20-28" come from if there is
a limit of 0-7. ( The 20-28 is actually 9 luns, maybe
that was a typo? )
3. Later on, they discuss "connection offsets". Maybe that's
just different terminology than what I'm used to. But I'm
really in the dark on what those are.

A little background:

I have seven Sun servers connected to an EMC Symmetrix. We have
LUN numbers going up to 255 in that environment. I have 4
HP-9000 D series servers connected to a different ( rather
old ) EMC Symmetrix. In the HP-9000 environment, all the luns
are 0-7. I had always assumed ( I know... ) that was a limitation
of the older Symmetrix.

We are planning to replace our D series servers with four rp34x0
servers. Two would be connected to the newer Symmetrix and the
other two would be connected to a new MSA-1000. The two sets
of servers are going to be at different sites, that's why they
will be connected to different arrays.

I'm concerned that I may need to do some reconfiguration of the
Symmetrix that I'll be connecting the rp34x0's to.

Thanks for any and all comments,
Tom

[Jeff Schussele · Accepted solution]

Hi Tom,

The "rules" as I know them are:

1) c can be from 0 ==> 255
2) t can be from 0 ==> 7
3) d can be from 0 ==> 7

HTH,
Jeff

[Tim Nelson]

Hey Tom,

Some of my comments were driven off of the way HPUX assignes HW addresses when using Symmetrix. The VBUS, TARGET, LUN. So the assignment of LUN number on the Symm is restricted.

I.e. LUN 100 would show on the server as
1/10/0/0.98.13.0.1.0.0 and /dev/dsk/cxt0d0( or something like that)(the x will be the next instance number when the bus is scanned)
LUN 0A1 would show as 1/10/0/0.98.13.0.0.10.1
/dev/dsk/cyt10d1 (the y will be some next instance number again)
I am only aware that the lun ( lower case as this would be the lun indicated on the hp side as in cxtydz <-- ) has a limit of 0-7. I have targets defined up to F.
ECC restricts the luns first bit to 0-7. I believe symconfigure will also.

Now the big question. What does StorageWorks do ? Uwe seems to have some experiences with StorageWorks and perhaps he can explaing.
There are a number of threads out here with the same issues but no resolutions posted yet.

[TwoProc]

Tom,

The Luns are larger than 7 on my server! However, the last number in the c0t0d0 (the d0) varies in value from 0-7. To handle the increase when you get to LUN8, the "t" value will increase by one (e.g. c0t1d0).
So,
c0t0d6 = LUN6
c0t0d7 = LUN7
c0t1d8 = LUN8
c0t1d9 = LUN9
etc.

[TwoProc]

Arrgh,
Someone was talking to me at the same time I was typing my response (bad multitasking) - so I goofed up.
Let me restate...
c0t0d6 = LUN6
c0t0d7 = LUN7
c0t1d0 = LUN8
c0t1d1 = LUN9
etc.

Think of it as two digits available for octal between the t and d numbers.

[marie-noelle jeanson_1]

Tom,

The reason for all that was explained above is that HP-UX only supports SCSI-2 addresses (in current releases). That means, it needs to create a virtual target for every 8 LUNs.
SCSI-2 defines 8 LUNs per target, therefore, every eighth LUN on the target will map to LUN 0 for a new SCSI-2 target. This explains in particuliar John' s example.

Marie.

[Tim Nelson]

Take a look at Dieter's thread.
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=952270

His issue is that lun numbers above 7 do not show up on the system. He can see 1 or 2 or 3 but not 8 or 9 etc..

No auto target increase in his case.

Comments ?

[Tom Dawson]

Everyone,

Thanks for the replies! I'll defer point assignment until this evening to keep the bunny off the thread.

In the mean time, I found on another thread that Mark van Hassel had posted a link to a HP Whitepaper on "Maximum LUN Limitations". I'm perusing that now.

Marie,

The concept of "virtual targets" had suprised me. Thanks for pointing that out.

Tom

[Ashwani Kashyap]

It depends on the addressing scheme used . But LUNS will always be only 0-7 .
HPUX has three addressing schemes
1. PDA - Peripheral device addressing .
2. VSA - Volume set addressing .
3. LUN - Logical Unit number addressing .

THe driver , upon detection of the fibre channel topology used switches to the appropriate addressing achemes . So depending on the topology you will see different addresses . Typically LUN scheme is used with normal SCSI controlloer , PDA with loop topology and VSA with switched fabric topology .

The VSA introduces more paths in the hardware addresses and some of the bits from those are used for the device addressing .
That is the reason VSA can address large number of LUNS .

See the chapter on fibre channel addresing in the atached PDF link .
http://www2.itrc.hp.com/service/iv/docDisplay.do?docId=prodITRC/DE_SW_UX_swrec_EN_01_E/FibreChannel.pdf

[Stuart Abramson]

4. LUN definitions:

Active LUN: Active. Load-bearing
Open LUN: Active but only occasionally used.
Visible LUN: Visible but not used.

5. Supported Limits:

HP-UX: 10.20 11.0 11i
ACtive LUNs: 768 2400 4096
Open LUNs: 1536 4800 8192
LUNs/HBA (non HA): 512 512 512
LUNs/HBA (HA): 320 320 512

(HA => MC/ServiceGuard or ServiceGuard-OPS)

6. Now let's do some arithmetic:

cCtTdD:
cC: 0-255 ?
tT: 0-15
dD: 0-7

15 x 7 = 105 PVs on a single "vbus"

7. VG Software limits:

256 VGs x 255 PVs/VG = 65,280

This is well ABOVE the recommended limit of 4096 ACTIVE LUNs.

8. GUIDELINES:

The effective limit of a big system is 4096 active LUNs:

a. That would be:

16 VGs of 256 LUNs.

b. Or:

128 VGs of 32 PVs (LUNs) each.

HP recommends a limit of 32 PVs per VG...

9. At boot or even during the day, ioscan can take a long time, the more
LUNs you have.

Install these patches to improve ioscan's performance probing LUNs:

HP-UX 11.0:
PHCO_24199
PHKL_24165
PHKL_24187

HP-UX 11.11:
PHCO_24199
PHKL_24163
PHKL_25165

10. EMC FACTORS:

The DMX series can only present 512 HVs to a single FA port.

You must set the FA to support HP-UX servers.
o Volume Set Addressing.

Targets vary from 0 - 15

LUN Numbers vary from 0 - 7.

You change the "vbus" to allow more LUNs.

[Uwe Zessin]

> 3. Later on, they discuss "connection offsets".
> Maybe that's just different terminology than what I'm used to.
> But I'm really in the dark on what those are.

It's not different terminology, but different mechanisms.

Dieter uses a storage array that is based on HSG80 controllers from the DEC/Compaq line.

The array-internal software uses a single address space for the mapping of a RAID storage-set to 'Units'. By default, the 'Unit' D0 is mapped to the hosts LUN address space as LUN 0, D1 as LUN 1 and so on. As has already been written - there is a problem for SCSI-2 drivers with Units/LUNs larger than 7.

A 'connection' is an array-internal object that describes the path from a server's Fibre Channel adapter to a specific controller port. It stores the controller and port, the adapter's WWN, the operating system and the 'offset' which was mentioned in the other thread.

Let's assume we set a connection's offset to 10. In that case, the first Units (D0..D9) are no longer accessible, D10 is mapped to LUN 0, D11 to LUN 1 and so on.


It is just a mechanism to be able to feed multiple servers using SCSI-2 drivers with different ranges from a single linear Unit address space.

It might look a bit complicated, but this technology is 7+ years old! (Took the first HSG training in october 1998)

[Tom Dawson]

Evryone,

Thanks for the replies! Obviously, I now need to start reading up on configuring a MSA-1000. The Symmetrix will be configured by our EMC CE.

Tim,

I understand what you are pointing out. It will be some time before I can experiment with it though. I'm not going to change the configuration on our existing HP-9000s, and it will the end of September before the new ones are in.

Thanks,
Tom

[Florian Heigl (new acc)]

In addition to Jeff's posting, I think it's
c[0-255]t[0-15]d[0-7], as I've met some ID 15 scsi devices on my test box, but I'm not sure this still applies with the virtual-something-mappings, maybe it only applies to local scsi targets.

florian