amount of disk devices

[Anthony Pratico]

Can anyone tell me, how many disk devices can an Openvms cluster system have...example DGA0: - DGA99999: ??? as in fibre attached storage...
I also have heard that Openvms systems can only have 256 disk devices per HBA is this true?

[John Gillings]

Anthony,

>how many disk devices can an Openvms
>cluster system have

Simple answer - "more than the budgets of most countries could ever afford to purchase"

or "more than 26" :-)

Suffice to say, if you have sufficient funding to break any existing limits on storage capacity, OpenVMS engineering will fix them!

[Jon Pinkley]

Is this a rhetorical question, or do you really intend to have more than 256 devices?

The maximum unit number is less than 99999, it used to be 9999, but now is closer to 2^15 (around 32000) with appropriate sysgen parameter values. But there are many other limits on numbers of devices, for example what the controller will support.

Can you tell us what limit you are hitting, or why you are asking the question?

Jon

[Hoff]

"How High is Up" questions are tough ones to answer. Mind providing some background on the particular request? Is this a serious inquiry, a question from a test or from an RFQ or such, or just for grins and giggles?

OpenVMS can configure more disk devices than most any entity can reasonably afford and expect to access from a particular server. And if an entity does deign to configure above the limits, then OpenVMS management can likely be convinced to increase the limit.

Disks available per HBA vary by HBA. Typical SCSI HBAs, for instance, support seven or fifteen connections. Individual SCSI connections can be disks, or can connect to secondary adapters (eg: HSZ).

MSCP disk services -- host-served disks -- are limited to 9999 units, IIRC, per host.

Individual device prefixes are limited to 9999 or 32767 devices (depending on the setting of DEVICE_NAMING), though additional prefixes or additional allocation classes can often be configured.

Add in a cluster configuration, and the scale of available storage gets yet larger.

How about you tell us how much storage you need, and relevant considerations here such as budget and bandwidth -- and that lets us figure stuff like cable lengths and cabinetry and available PCI slots.

And when discussing large volumes of storage and per a CERN study of large-scale disk farms, assume roughly three hard errors per terabyte of disk storage. On average.

[Robert Brooks_1 · Accepted solution]

I also have heard that Openvms systems can only have 256 disk devices per HBA is this true?

--

No, there is no HBA-specific limit within the operating system. Steve's posting listed several other gating issues that would potentially limit the number of devices on a system, but the simple answer to your above-quoted question is no.

-- Rob (ex-VMS Engineer who spent a fair amount of time dealing with devices in general, fibre channel in particular)

[Anthony Pratico]

Now Yes, is there a limitation and yes I do have several disks. We are using XP12000 and I can create 500,000 drives + from this XP with the proper XP config.

The PROD SAN that we have currently will be shared with our disaster recovery system and HP is recommending doing SHAWDOWING to UPDATE our DRC. They reccomend this due to the OVERHEAD on the XP FRAMES with all of our IO...

So that being said, I am sure I did not explain it to the fullest, so my SAN will explode so to speak.
I need to designate segments of disks such as dga0 - dga30000 to production and dga30001 to dga60000 to DRC and if we merge more of our SANS from our development and INSTALLS clusters then I need to know the disk limitations.

And no we do not use MSCP serving.

We have prod 6 node cluster and a DRC 5 node cluster.

[Hoff]

I'm mildly surprised that a site with hardware of the scale of an XP12000 and that is considering 500,000 disk devices is here in ITRC; I've have expected to be working more directly with and HP on this configuration and on related Best Practices. There's a whole collection of factors here -- beyond the numbers of volumes -- that can come into play with a disaster-tolerant cluster.

If you want to bring 500,000 disks on-line for production, check with HP OpenVMS management. The XP12000 connects all of 1152 physical disks and approximately a third of a petabyte, so it looks like you're using some very small partitions here. I'd not expect to get more than 32,767 DG disks on-line by default, and I haven't seen indications of support for multiple allocation classes for FC disks. (And that's still analogous to having 32 partitions per physical disk, with no RAID and no replication overhead.)

Simply managing this quantity of disk devices -- just keeping this number of disks on-line and mounted and archived as appropriate on OpenVMS itself -- is going to be a significant challenge. I'm not aware of an OpenVMS mechanism akin to the Linux or Unix Logical Volume Manager (LVM), for instance.

Though you've probably already read this, here's the XP12000 manual for OpenVMS:
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01081855/c01081855.pdf

I'd suggest you ring up your local HP OpenVMS Ambassador or HP rep as a start.

Stephen Hoffman
HoffmanLabs LLC

[Anthony Pratico]

Hoff thanks but we are not considering creating 500,000 drives but with the XP we have the potential to do so....

My question is due to HP's response of SHADOWING drives between our 2 sites PROD and DRC...we have to unite the 2 SANS and I can't have drive DGA50 on prod shadowed with DGA50 on the DRC, because we cannot use the allocation class as you state.

What is my limit? is it 32767 drives that VMS can see...as in DGA0 to DGA32766?...Yes we are working with HP and they also told us that you can ONLY have 256 drives per HBA, which from this thread I am told that is false.

[Doug Phillips]

Anthony,

Listen to what HP has told you. Your limit is how many drives can be supported by the HBA, not by OpenVMS.

It's like the difference between "how much food can a restaurant prepare" and "how much food can you eat."

[Anthony Pratico]

Doug thanks for your response but I am not sure you get my question.....

An Openvms system can handle up to 32767 disk drives, regardless of how many HBA'S are used in the system box...

dga0:
dga1:
...
dga32766:

I am listening to HP and sometimes their recommendations work for most sites but not all.
Now that being said, If I have one 5 node cluster with it's own SAN, and another 6 node cluster with it's own SAN and I need to merge both sans so all nodes on all clusters and see all drives so I can utilize the VOLUME SHADOWING, I CANNOT have the same drive numbers from both sans on the same system, VMS frowns on that....

I need to know the limit so I can desginate a group of drives to PRD and another to our DRC and then volume shadow them.
The Highest unit used to be 99999 but now through these threads I am assuming that it still could be but the total amount of disk devices cannot exceed 32767.

[Bill Hall]

Anthony,

The 256 LUN per HBA is a documented limitation on the EVA series of arrays. The EVA arrays can present a maximum of 256 LUNs to a single HBA, IIRC the HSG80 based arrays were limited to only 128. Consider also that the EVA can present a relatively low total number of LUNS (1024???), but I consider it much more flexible and easier to manage than the XP arrays.

I have never seen a similar limitation published for the XP series of arrays.

Rob has verified that VMS does not impose a 256 device per HBA limit. Hoff has supplied the current unit number limits both MSCP served devices and non-MSCP served devices. Hoff also brings up a valid point about the manageability of a large number of small devices vs. a smaller number of large devices.

Bill

[Jan van den Ende]

Anthony,

another point of consideration.

This looks like big fiture planning.

Do _NOT_ assume that current limits are fixed!

Of course, on current hardware stay within the current limitations, but as Bill already remarked, those are different for EVA and XP.

So, _NOW_ design for future growth!

How to do that, requires MUCH more info, which you should have (to be able to DO the job.

Just ALWAYS keep in mind to NOT lock yourself out of future developments.

(when we set up our current cluster, nobody had heard of SANs, but we migrated towards using it without a single second of downtime).

Wishing you success, and much wisdom.

Proost.

Have one on me.

jpe

[Anthony Pratico]

Thanks Guys for you input, you are correct this for for now and future planning and growth, and of course I do not want to lock out of any future changes.

By doing what we and HP are planning, we are trying to minimize or eliminate the perceived downtime updating our DRC and to actually have as close to a HOT site as possible.

Thanks All!

[Doug Phillips]

Anthony,

My point was pretty obscure, I guess: You need to be looking at real numbers. Like actual storage volume rather than than just number of disks. Do you really expect to have more volume than can be stored on 32K disks? What size are your disks and partitions? Should you maybe be looking at larger disks?

Use a realistic allocation of the DG assignments. You want to group them by production and DRC allowing for 30K each? Is that realistic? Would 10K or 15K per-group be enough? If not, then, you need to go another direction (as mentioned by others.) If you're really talking about that size need, then you must have a *very large* budget.

How much growth do you *really* expect to have in the future; say 3 to 5 years? Double that number if you want to. Then, decide if you need to take another approach. Also consider the advancements in storage technology that we might see in the future. Chart the curve back 5 years and project it forward to get the idea.

The fact that the XP12000 *can* create 500,000 drives is irrelevant if you are never going come close to using that many. If you do expect growth that will exceed 32K per DGA and 256 disks per HBA, (I'd love to see that installation when you get it up and running!) then look at other options.

[Guenther Froehlin]

We aer not talking about max disk unit numbers, aren't we? Like a scheme to number shadow set members nicely by using high digit numbers to identify them as being shadow set members of the same shadow set?

/Guenther