OpenVMS driver in C

[Alex Chupahin]

Hello!
Is there a tutorial how to writing drivers for OpenVMS in C? Or are there examples of simple drivers especially tapes?
Please do not touch me to sys$examples - this driver is TOOOO complex and not clear very much.
Also please do not touch me the book "Writing OpenVMS device drivers in C" This book is a API table and not a tutorial. Not useful just now.

[Hein van den Heuvel]

Is there a tutorial how to writing drivers for OpenVMS in C? Or are there examples of simple drivers especially tapes?

Alex, There is no such thing as a simple driver!

Hope not to insult you by asking: Are you sure you need a driver and not just basic IO functions like SYS$ASSIGN, and SYS$QIO and stuff as described in the
"HP OpenVMS I/O User's Reference Manual" ?
http://h71000.www7.hp.com/doc/732FINAL/aa-pv6sf-tk/aa-pv6sf-tk.HTMl

Does the tape have a unique (pci) interface card, or is is 'just' a scsi tape connected to 'standard' scsi controller (KZPxx)... ?

>> please do not touch me the book "Writing OpenVMS device drivers in C" This book is a API table and not a tutorial.

Dang... that's exactly the book I'd recommend. That is the 'Margie Sherlock / Lenny Szubowiz' right?

hth,
Hein.

[Robert Gezelter]

Alex,

The short form of the general restriction is that the C-run time support is very limited, since the driver is in kernel mode.

- Bob Gezelter, http://www.rlgsc.com

[Hein van den Heuvel]

Oh wait...

Is this in reference to:

http://forums12.itrc.hp.com/service/forums/questionanswer.do?threadId=1184845

Writing a device driver, from scratch, to properly work with an unsupported SCSI card is about 1,000 times more interesting, 100,000 times slower, and 1,000 times more expensive than buying / borrowing a supported interface.

If you do decide to go this route, for educational purposes, then please lean on OpenVMS Engineering friends (come to the Bootcamp!?) or supports put a request on openvms.org and/or c.o.v and/or decuserve for a copy of the current SCSI drivers.
Allthough... that's probably not be in C.

fwiw,
Hein.

[Jon Pinkley]

Alex,

I can only say that if you think the sys$examples:lrdriver.c is too complex, it is simple compared to most drivers. SCSI drivers have to deal with many more special cases than a parallel port driver. Although the size alone is not necessarily a good indicator of complexity, look at the sizes of the lrdriver vs the SCSI port drivers (pk).

This is from 7.3-2

$ dw sys$loadable_images:*lr*

Directory SYS$COMMON:[SYS$LDR]

SYS$LRDRIVER.EXE;1 31/32 1-OCT-2003 21:19:39.98

Total of 1 file, 31/32 blocks.
$ dw sys$loadable_images:*pk*

Directory SYS$COMMON:[SYS$LDR]

SYS$PKADRIVER.EXE;1 713/720 8-JAN-2007 16:38:02.90
SYS$PKCDRIVER.EXE;1 171/176 1-OCT-2003 21:19:28.17
SYS$PKEDRIVER.EXE;1 193/200 1-OCT-2003 21:19:29.75
SYS$PKJDRIVER.EXE;1 144/144 1-OCT-2003 21:19:31.82
SYS$PKQDRIVER.EXE;1 276/280 1-OCT-2003 21:19:35.64
SYS$PKRDRIVER.EXE;1 261/264 8-JAN-2007 16:38:01.92
SYS$PKSDRIVER.EXE;1 196/200 1-OCT-2003 21:19:32.31
SYS$PKTDRIVER.EXE;1 195/200 1-OCT-2003 21:19:34.52
SYS$PKWDRIVER.EXE;1 238/240 8-JAN-2007 16:38:02.35
SYS$PKZDRIVER.EXE;1 171/176 1-OCT-2003 21:19:34.91

Total of 10 files, 2558/2600 blocks.
$

If you do decide to go down this route, you will need to have a dedicated box to develop on, and you will become very familiar with the SDA (analyze/crash) utility before you have a driver that works. And then you will be at the stage where you start to fix it so it works well, supports tag queuing correctly, etc, etc.

And the SCSI driver in VMS are divided into port (specific to the adapter hardware) and class (specific to the device type, i.e. DK, MK, GK, etc). That's another level of complexity (but it is the right way to do it so you don't need to write a driver for every adapter/tape, adapter/disk etc.)

Look for a used supported card if the budget is tight. With many people going to fibre channel, you may be able to get them quite cheap, although getting one shipped to your location may cost a bit.

Good luck,

Jon

[Alex Chupahin]

> Alex, There is no such thing as a simple driver!

Yes of course. But please look to a Unix/Linux world. There are a tutorials for a way from simplest driver/module for ten strings that do a thing like "hello world" to a large driver for real device.
So I can now write a linux device driver and understand concepts.
I'd like to see such thing in VMS.

>Are you sure you need a driver and not just basic IO functions like SYS$ASSIGN, and SYS$QIO

yes. It should be "virtual" tape driver without a real device. Similar to LD.

>That is the 'Margie Sherlock / Lenny Szubowiz' right?

Yes. This is a book is an API table without describing mechanism clearly. So I am
disappointed.

[Jon Pinkley]

RE: "yes. It should be "virtual" tape driver without a real device. Similar to LD."

Oh, you mean like LD V9? That does exactly that.

See Jur's site: http://www.digiater.nl/

Jon

[Alex Chupahin]

> Is this in reference to:
No no no :)

It is learn purposes with little practise use:
I need a "virtual" tape sequental driver.

> I can only say that if you think the sys$examples:lrdriver.c is too complex, it is simple compared to most drivers.

I'm sure real device drivers are much more complex then in sys$example one.

But I wish to learn driver architecture.
First, I wish to see a driver todo nothing like NUL: ! It can only loads and shows his status.
Second, example with little more functionality: it may be opened by sys$open
and write to console "I'm super driver"
end so on.

Is there any tutorial like this?

[Alex Chupahin]

>Oh, you mean like LD V9? That does exactly that.

Not exactly as I wish.
My driver should create a file with size of 0 blocks and append it. looks like TAR but should be a driver.

[Jon Pinkley]

RE:"My driver should create a file with size of 0 blocks and append it. looks like TAR but should be a driver."

So what you really want is a TARACP (similar to what the MTAACP does for File11 tapes), or what the XQP does for disks.

ACPs sit between the driver's FDT and the start i/o routine. They do processing below IPL 3, they operate in process context instead of driver context, so they can do things like call system services.

As far as I know, there isn't any official documentation on writing Ancillary Control Process. Jamie Hanrahan gave several presentations at DECUS (I have the notes from his 1990 Spring DECUS "The What, Why and How of User-Written ACP's"). He and Lee Leahy wrote a book "VMS Advanced Device Driver Techniques" 1988, long out of print . It had had a chapter that was very similar to the notes from his 1990 DECUS presentation. That was for the VAX, but the principles are the same.

Here's an excerpt
---------
Why not use an ACP?

ACP implementation and maintenance requires considerable expertise (same technical level as driver writing, but additional knowledge is required)

ACP's use system routines, data structures, and other kernel-mode interfaces that are not described in the VMS documentation and that must therefore be considered subject to change in future releases of VMS.

Sending a request through an ACP introduces considerable additional overhead.
---------

It is hard for me to understand why having a TARACP would be that beneficial. But you did say this was for learning purposes.

Here are some things to search for with Google if you want some examples.

Glenn Everhart had an ACP virtual disk driver that was submitted to a DECUS tape.

There is a "network tape driver" that I believe is ACP based. It allows a remote tape to be accessed over a DECnet connection, if I am remembering correctly (I never used it).

Just to be clear, LDDRIVER is what I would call a front end driver. It has the same basic parts as a real driver on the user facing side. Its FDT routines accept the requests, modify the LBNs but the start i/o routines hand the request to another driver's routine.

In concept it is somewhat like what using NAT to provide a virtual HTTP server. NAT provides a different view of the same IP address.

As far as your request for a "simple" explanation of i/o in VMS, there was a (2?) hour session given at the last VMS bootcamp by one of the VMS developers.

I agree that making more of that type information publicly available would be a good thing.

But don't expect that after a 2 hour session (or a one week intensive class) you are going to be ready to write a good quality SCSI port driver. Drivers are extremely complex, and there are many pitfalls to avoid.

VMS drivers by design are written to block device interrupts for a minimum amount of time, and then continue on at a lower IPL (fork level) so other device interrupts can interrupt the fork level processing. This is good for reducing the interrupt processing latency, but it is a pain for the device driver writer. Data structures have to be synchronized, etc. etc. It makes writing threaded code look simple.

And to top it off, it has to be correct. If it isn't, you crash.

Jon

[Robert Brooks_1]

What, really, are you trying to accomplish that cannot be done with some other (existing) mechanism?

I've read your response of

"My driver should create a file with size of 0 blocks and append it. looks like TAR but should be a driver."

It's still not clear (to me) how you want to do this. Do you want to replace the existing SCSI stack on VMS? Do you want to use some never-before connected device? There may be more easily-attainable ways to do this.

The Sherlock/Szubowicz book is quite good at what it purports to do, which is explain the nuts and bolts of writing a VMS device driver in C. The book is largely aimed at those who
already understand the VMS driver environment and attempts to document the kernel routines one would use. It's not for someone who doesn't already understand things like IRP, UCB, DDB, DDT, FDT, STARTIO, KP_STARTIO, etc ...

If you've got the source, take a look at DQDRIVER (the IDE driver). It's a monolithic class/port driver.

As said before, if you think LRDRIVER is too complex, then you are likely over your head already.


-- Rob

[Hoff]

There's the old Step-Two driver documentation, and the old driver manuals were divided into tutorial and reference books. These manuals well predate Margie and Lenny's Driver book, and the latter book is showing its age. These manuals are in the archived documentation section, IIRC.


Here's an I/O intro:
http://64.223.189.234/node/114

Notes on driver debugging (started this just yesterday; very preliminary)
http://64.223.189.234/node/753

An older version of DQDRIVER is on Freeware V6.0.
http://64.223.189.234/node/750

Ancillary Control Processes (ACPs):
http://64.223.189.234/node/213

(I wrote my first ACP a couple of months before Lee & Jamie published the book. Grumble. I do have a copy of the original book, picked up when a corporate library closed and dispersed its books.)

Getting from kernel-mode to disk is going to involve either an ACP, or some fancy footwork with mapping the file akin to what Jur does with LD. RMS and kernel-mode at IPL are incompatible, hence the ACP.

Of the various OpenVMS ACPs present in the source listings, MTAACP is one of the easiest to read. Stay away from the DECnet ACPs; those are really twisty, due to the way they have to work.

[Alex Chupahin]

> The Sherlock/Szubowicz book is quite good at
>what it purports to do, which is explain the
>nuts and bolts of writing a VMS device driver
>in C. The book is largely aimed at those who
>already understand the VMS driver environment
>and attempts to document the kernel routines
>one would use. It's not for someone who
>doesn't already understand things like IRP,
>UCB, DDB, DDT, FDT, STARTIO, KP_STARTIO, etc

True.
So where I can study what is DDb,DDT, etc,etc,etc with clear examples of using?

Unfortunately I cant be at the Bootcamp (it is VERY hard for me, or impossible even).
May be, it is a good idea to put examples from VMS guru in the web if such theme will be in the meeting? I'm sure not only I am is interested in it.
NL: driver (/dev/null) is a good start may be.

I dont know can ACP help me.
I wish to create a device named, VTP: for example, then I'll can do:
$COPY file.ext VTP0:
or
$BACKUP [...]*.* VTP0:

[Jur van der Burg]

I've had a request for LD virtual tape support to automatically extend a file when needed, or start from a zero size. I will look at that for a future version of LD.

If you think that LRdriver in sys$examples is difficult then you may want to re-think your approach. You may be better of with a simple skeleton driver and an acp, or just plain usermode application code. A driver can be very efficient for heavy duty use, but it may not be the best solution. There are other ways.

I've been writing VMS drivers for more than 25 years and it's easy once you know the inner workings of VMS. :-). But it takes time to learn that.

Jur.

[Jon Pinkley]

>>"So where I can study what is DDb,DDT, etc,etc,etc with clear examples of using?"

How about "Writing OpenVMS device drivers in C" chapter 2, specifically look at 2.8 Example of an I/O request" starting on page 19.

After you read chapter 2, then look at lrdriver.c again, and see if you can then follow what is in it.

What will you do with this "virtual tar tape" once you have it? And why can't you use something like VMSTAR instead? If you are looking for something to exchange files between Linux and VMS, why not use zip/unzip?

Jon

[Robert Gezelter]

Alex,

Put simply, an ACP does processing that it is possible (or at least not easily possible) from within the limited context of a driver.

Drivers operate effectively as subroutines of the executive, not independent processes. As Hoff has noted, for example, a driver cannot directly use RMS or for that matter, the file system.

An ACP (ancillary control process) on the other hand, is a full, albeit privileged, process. Its function is to take an IO Packet and do some supplemental (or complete processing) to it. For many years, all file system processing was done using F11BACP, the Files-11 Level 2 ACP. This was reimplemented as the XQP to improve performance. On magnetic tape, the ANSI processing is performed by the ACP.

In your case, the ACP could accept the packet from the driver, do the necessary processing (including, for example, any buffering or time consuming compression calculations), and then signal the completion of the operation. It would also write buffers to the actual file.

- Bob Gezelter, http://www.rlgsc.com

[Willem Grooters]

There used to be a book "Writing device drivers in C" that came with an earlier version of VMS (7.3). I don't have the details at hand at the moment.

WG

[Jon Pinkley]

These documents are the extent of external documentation for device drivers in VMS VAX and Alpha (aka AXP)

They are in the archived documents page http://h71000.www7.hp.com/doc/archived.html with the exception of the Digital Press book "Writing OpenVMS Alpha Device Drivers in C", which was only released in printed form (that I am aware of).

All of the following pdf documents were released in 1994 with VMS 6.1

As such they are quite dated, but the basic principles are the same. At that time VMS on Alpha was essentially a 32 bit operating system, the high longword of addresses were sign extended from the lower 32 bits.

All of the device support manuals are written for system's programmer's, i.e. you are expected to understand VMS internals, how synchronization is maintained with IPL and spinlocks, etc.

The "Writing OpenVMS Alpha Device Drivers in C" book was released in 1996, and describes the VMS 7.0 environment, including 64 bit address space. But many things have been added since 1996 and 7.0. As far as I am aware, there is no external documentation on writing device drivers for the IA64.

----------

OpenVMS VAX Device Support Manual

http://h71000.www7.hp.com/doc/73final/documentation/pdf/OVMS_VAX_SUP_GD.pdf

OpenVMS VAX Device Support Reference Manual

http://h71000.www7.hp.com/doc/73final/documentation/pdf/OVMS_VAX_SUP_REF.pdf


OpenVMS Alpha Device Support: Developer's Guide

http://h71000.www7.hp.com/doc/73final/documentation/pdf/OVMS_DEVGD_VMSINSTAL.pdf

OpenVMS Alpha Device Support: Reference

http://h71000.www7.hp.com/doc/73final/documentation/pdf/OVMS_ALPHA_DEV_SUP_REF.pdf

Creating an OpenVMS AXP Step 2 Device Driver from a Step 1 Device

http://h71000.www7.hp.com/doc/73final/documentation/pdf/CR_AXP_STEP2_DR_FR_STEP1.pdf

Creating an OpenVMS AXP Step 2 Device Driver from an OpenVMS VAX Device Driver

http://h71000.www7.hp.com/doc/73final/documentation/pdf/CR_AXP_STEP2_DR_FR_VAX_DR.pdf

[Keith Cayemberg]

Hi Alex,

have you looked at the Archived Manuals at the OpenVMS Documentation Site?
http://ftp.openvms.compaq.com/doc/archived.html

OpenVMS AXP Device Support: Developer's Guide
Describes how to write a driver for OpenVMS Alpha for a device not supplied by Compaq.

OpenVMS AXP Device Support: Reference
Provides the reference material for the Writing OpenVMS Alpha Device Drivers in C by describing the data structures, macros, and routines used in device-driver programming.

OpenVMS VAX Device Support Manual
Describes how to write an OpenVMS VAX driver for a device not supplied by Compaq.

OpenVMS VAX Device Support Reference Manual
Provides the reference material for the OpenVMS VAX Device Support Manual by describing the data structures, macros, and routines used in device-driver programming.

Creating an OpenVMS Alpha Device Driver from an OpenVMS VAX Device Driver
Describes the procedures for converting a device driver used on OpenVMS VAX to a device driver that runs on OpenVMS Alpha. This book also contains data structures, routines, and macros for maintaining an Alpha driver written in Macro-32.

Creating an OpenVMS AXP Step 2 Device Driver from a Step 1 Device Driver
Provides information for upgrading a Step 1 device driver (used in earlier versions of OpenVMS AXP) to a Step 2 device driver. A Step 2 device driver is required for OpenVMS AXP Version 6.1.

Creating an OpenVMS AXP Step 2 Device Driver from an OpenVMS VAX Device Driver
Provides information for migrating a device driver used on OpenVMS VAX to a Step 2 device driver used on OpenVMS AXP Version 6.1.

I took the above descriptions from the following URL...
http://ftp.openvms.compaq.com/doc/83FINAL/6679/6679pro_014.html


Other potentially relevant URLs...

VAX/VMS Realtime Programming
An Application Note by Bill Forbes
http://www.ibiblio.org/pub/academic/computer-science/history/pdp-11/hardware/micronotes/numerical/micronote44.txt

AXP-DRIVER-EXAMPLE
http://h71000.www7.hp.com/freeware/freeware80/AXP-DRIVER-EXAMPLE/


Cheers!

Keith

[Jon Pinkley]

RE:"VAX/VMS Realtime Programming
An Application Note by Bill Forbes
http://www.ibiblio.org/pub/academic/computer-science/history/pdp-11/hardware/micr
onotes/numerical/micronote44.txt"

This is in interesting read from a historical perspective, as it was written over 21 years ago (in 1986). One thing described is the connect to interrupt driver, which on the VAX allowed you to redirect a hardware interrupt to your code, without having to write a device driver. That functionality never moved to the Alpha, i.e. there is no connect-to-interrupt driver on Alpha. So you won't be able to use that as a shortcut for a simple driver.

Jon

[Alex Chupahin]

Thank you very much for all,
I'll try to study this information.

[anonys]

Hi Alex,
whether you find the most appropriate example? Can share the experience about it?thank you!

[Hoff]

Anonys, you are in an OpenVMS operating system forum, and OpenVMS drivers are wildly different from the HP-UX drivers (and Unix drivers in general) that are the subject of your other recent question:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1252977

If you want to see what an OpenVMS device driver looks like, gain access to an OpenVMS system and look for this file specification:

SYS$EXAMPLES:*DRIVER.*

The Deathrow cluster has open access to OpenVMS; you can sign up for a free account over there. HP offers access to various test drive systems, each of which also has free login accounts available.

There are also driver examples in the OpenVMS device driver documentation -- the older documentation stuff is available in the archived documents section in the left navigation of the OpenVMS documentation web site:

http://www.hp.com/go/openvms/doc/

The newer driver manual (writing drivers in C) is not available on-line AFAIK. It's a for-purchase book.

There are various OpenVMS device drivers available in source code form on the OpenVMS Freeware and the other similar resources located at:

http://mvb.saic.com/

You can use Google site:mvb.saic.com to target a Google search against that site.

And again and having written drivers for both OpenVMS and for various Unix platforms, the OpenVMS device drivers are completely different than Unix drivers.

Would you be seeking assistance on some sort of formal training or class assignment or homework here?

[Bill Pedersen]

Am interested in getting my hands on the most recent version of "Writing OpenVMS Alpha Device Drivers in C" I understand there was a second 2nd Edition but when I have tried to find it from a technical bookstore out in San Diego they say it was never published. Does anyone have any better information? My current copy (version unknown) is in storage in California so am looking to get one since I have relocated to the East Coast.

Thanks,

Bill.

[Robert Brooks_1]

Am interested in getting my hands on the most recent version of "Writing OpenVMS Alpha Device Drivers in C" I understand there was a second 2nd Edition . . .

--

I can say with absolute certainty that a 2nd edition was neither written nor published.

I can say with near certainty that a 2nd edition will not be written.


-- Rob