sys$qiow parameters

[Rizwan Latheef]

Iam using IO$_READVBLK to read the message from the TCP/IP socket.
status = sys$qiow(3,
channel,
IO$_READVBLK,
&iosb,
0,
0,
message,
sizeof message, 0, 0, 0, 0)) & 1))
For example I define the message size as 8k or 8192 bytes and if the actual message is more than that what will happen? Will it pull exactly 8k and I will have to do an another sys$qiow call to pull the rest or will it pull just 8k and the rest is lost?

[Robert Gezelter]

Rizwan,

I have done this many times.

Extreme caution is recommended. TCP sockets are stream-oriented devices and the IO operations to them do not make any guarantees about the data returned, except that it will be less than or equal to the length specified.

Message framing is the responsibility of the application, not the IP stack (This is a DRAMATIC difference between IP and DECnet. DECnet IS generally message oriented).

TCP messages can be broken into multiple segments at any time and for any reason. Applications-level code must determine if a complete message has been received, or if additional IO operations are required).

- Bob Gezelter, http://www.rlgsc.com

[Guenther Froehlin]

No, the rest is not lost. It is buffered in system space. Just keep on reading from your input stream and compile your message format from the pieces you read in.

/Guenther

[Hoff]

You're sipping from the TCP byte stream here. You're not working with datagrams. Not Ethernet frames. Not VMS-style records. You have a flow of bytes arriving.

You'll typically read one or more bytes each time you sip. You might even get more than a full message if multiple messages are queued up and if your read request is larger than one message. Or you might get one byte, and have to build out a message a byte at a time.

What "fools" many TCP programmers early on is that they usually do get the datagrams; that the message blocks arrive as expected. Until the pipe gets stuffed up or something somewhere gets a little bollixed in the IP plumbing, or software or routers or such things get loaded or stressed out. Then "weird stuff" -- which is actually the expected and normal TCP stream processing -- starts happening to the application.

(And I've certainly been there and made this coding mistake with TCP some eons ago; I'd ended up with a "torn" message under load -- the first part of the next message, and the last part of the previous message lurking in the remainder of the buffer -- and the application went deep into the weeds. Took a while to find that one.)

[Robert Gezelter]

Rizwan,

Hoff is quite correct.

It is also conceivable that one can receive a zero length response IF the timing is JUST right. Forewarned is forearmed.

The code to retrieve bytes from the stream is separate from the code that extracts messages from that stream. With TCP, there is no way around having to implement a record framing element of the program.

- Bob Gezelter, http://www.rlgsc.com

[Rizwan Latheef]

Thanks Guys. It worked. Indeed it is true that the balance of the messages do stay in there and I have to do an another QIOW call to get the whole stream of bytes that Iam looking for.

[Willem Grooters]

Just for confirmation.
1.
FWIK, the TCP protocol standard describes a message is guaranteed to be delivered IN TOTAL and IN SEQUENCE. That means, that if a message is split up in packages, the receiver may assume the messsage is completely delevered with packages in sequence.That means that for the receiving program, there will be no interleaving packages, all be in sequence.
This opposite to UDP, where this is not the case, and the receiver has to deal with missing, incomplte, interleaving, and out-of-sequence packages.
2.
IIRC, QIOW - not QIO - will wait until the buffer is full OR an 'end-of-data' is reached. If QIOW returns, the buffer is full, or the message is complete.

Having said that, is the buffer is full, QIO will return with it's buffer if 8K filled up, and the remainder of the emssage will still be available, to be read with the next invokation.

[Robert Gezelter]

Willem,

My earlier posting may not have been phrased as clearly as it could have been.

TCP guarantees an uninterrupted (reasonably free of errors) stream. The QIOs will return the next group of bytes in the buffer, in sequence. There is no way to know if that next group of bytes is:
- part of a message
- an entire message
- an entire message and part of the next message
- the ending part of a message and the beginning of the next

These possibilities are a consequence of the architectural decision to make TCP a "stream" protocol. While I have not re-read the documentation recently with an eye towards this question, I would also presume that it is possible to get a zero byte count in some cases (other than the socket being closed).

In all cases, my recollection is that the various OpenVMS stacks all return the byte count of the result in the IOSB.

My comment "reasonably free of errors" is a deliberate reference to the limits on the checksums underlying the TCP protocol. If error control is a serious concern, applications-level cryptographic checksums are highly recommended; for lesser purposes, wider CRCs are also useful (details of the TCP protocol are available in the current RFC which is available at http://www.ietf.org ).

- Bob Gezelter, http://www.rlgsc.com

[David Jones_21]

"2.
IIRC, QIOW - not QIO - will wait until the buffer is full OR an 'end-of-data' is reached. If QIOW returns, the buffer is full, or the message is complete."

To complicate matters, the tcpip services driver has a TCPIP$C_MSG_BLOCKALL flag you can specify in P5 that forces the QIO/QIOW to keep reading until the buffer is full or the connection is closed. It can simplify the logic a little in cases where the application protocol has fixed size messages or messages with a fixed header followed by a variable number of bytes indicated within that header.

[David Jones_21]

Oops. The read attributes mask is specified by P4, not P5.