topic Re: 11382 byte record too large for user's buffer in Operating System - OpenVMS

11382 byte record too large for user's buffer

A.W.R — Mon, 13 Jun 2011 13:08:09 GMT

Hi,

We have received a file from a UNIX system. This file is transferred as one continuous string. When we try and open the file from DCL we get the error:-

%RMS-W-RTB, 11382 byte record too large for user's buffer

The code is:-

$ >>TYPE EXAMPLE_JSA.DAT;1
%TYPE-F-WRITEERR, error writing SYS$OUTPUT:.;
-RMS-F-SYS, QIO system service request failed
-SYSTEM-F-EXQUOTA, process quota exceeded

ITP001>>open/read infil EXAMPLE_JSA.DAT
ITP001>>read infil inrec
%RMS-W-RTB, 11382 byte record too large for user's buffer

The process quota are:-

Show proc/quota

CPU limit: Infinite Direct I/O limit: 2000
Buffered I/O byte count quota: 255616 Buffered I/O limit: 2000
Timer queue entry quota: 100
Open file quota: 199
Paging file quota: 1592976 Subprocess quota: 20
Default page fault cluster: 64
AST quota: 298
Enqueue quota: 4000 Shared file limit: 0
Max detached processes: 0
Max active jobs: 0

Any suggestions on how we can manage this file from the OpenVMS side would be greatly appreciated.

Thanks
Andrew

Re: 11382 byte record too large for user's buffer

Steven Schweda — Mon, 13 Jun 2011 13:48:15 GMT

Uh, don't use TYPE?

> We have received [...]

How, exactly?

> [...] a file [...]

What's in this "a file"? Text? Non-text?

What, exactly, would you like to do with
these data?

> When we try and open the file from DCL [...

To me, "open the file from DCL" says "OPEN",
not "TYPE".

> [...] how we can manage this file [...]

Define "manage".

Re: 11382 byte record too large for user's buffer

A.W.R — Mon, 13 Jun 2011 13:51:40 GMT

HI,

Thank you for your response.

We did try the "OPEN" command from DCL as per the original post, but this did not work either - see above.

The data in the file is text.

We need to access the data.

Thanks
Andrew

Re: 11382 byte record too large for user's buffer

Duncan Morris — Mon, 08 Aug 2011 20:22:48 GMT

Hi Andy,

have a look at this thread for a similar discussion.

http://h30499.www3.hp.com/t5/General/10428-byte-record-limit/m-p/4746130#M19306

There you will see the size limits for DCL reads, and for high level language RMS GET operations.

Try showing the output of $DIR/FU for the file EXAMPLE_JSA.DAT

Duncan

Re: 11382 byte record too large for user's buffer

Steven Schweda — Mon, 13 Jun 2011 13:58:13 GMT

> Thank you for your response.

So, where are the answers?

> > We have received [...]
>
> How, exactly?

Then, of course, "DIRE /FULL EXAMPLE_JSA.DAT".

> The data in the file is text.

How long are the lines?

Re: 11382 byte record too large for user's buffer

Joseph Huber_1 — Mon, 13 Jun 2011 14:07:07 GMT

If the file is really human readable text, then it should not have 11382 byte records!

What means "This file is transferred as one continuous string" ?
Which transfer method ? (FTP ascii or binary ?, HTTP ?).
If the file is already on Unix in such a form (i.e. one long stream of bytes without line terminators), then there is no way to handle it with TYPE, VMS editors or DCL OPEN/READ. You have to use an application program which knows how to handle it.

Re: 11382 byte record too large for user's buffer

Hoff — Tue, 02 Aug 2011 14:07:45 GMT

There's no such thing as a simple sequential text file.

There's also no canonical file transfer sequence.

And there are bugs in the handling of record lengths in some older versions of VMS, and variously around what turned out to be errant LRL longest record length settings.

Further, older versions of VMS are rather worse here in terms of the buffer sizes themselves. If your unspecified version of VMS is older than V7.3-2 here, then please consider upgrading to at least that revision and (due to various other DCL- and RMS-related changes) to V8.3 or so.

As for the usual error: Ill-constructed ftp transfers are notorious for stomping on RMS metadata, and for hauling over files that are unreadable.

Here is a write-up on sequential file formats and ftp file transfers:

http://labs.hoffmanlabs.com/node/302

Some of the other existing discussions available via Google:

http://h71000.www7.hp.com/wizard/wiz_7161.html

http://h30499.www3.hp.com/t5/Languages-and-Scripting/RMS-W-RTB-when-execute-DCL-script/m-p/5174778#M11800

http://h71000.www7.hp.com/wizard/wiz_2682.html

http://h30499.www3.hp.com/t5/System-Management/record-too-large-for-user-s-buffer/m-p/4343396#M23118

http://h30499.www3.hp.com/t5/System-Management/ZIP-amp-UNZIP-1-9-Aug-26th-1992/m-p/4770826#M28422

http://h30499.www3.hp.com/t5/System-Management/Error-executing-com-script/m-p/4377026#M23708

http://h30499.www3.hp.com/t5/General/10428-byte-record-limit/m-p/4746130#M19306

---

And FWIW, that quota stuff you attached? None of these quotas typically applies to an application buffer size error. If you should hit a process quota error, it'll usually either stall (and a stall which usually clears itself as the quota then becomes available), or you'll get an explicit quota-related error or (if the quotas are massively over-assigned and over-allocated) VMS itself will become unstable. But you won't usually get an RTB here.

---

$ help/message/facility=rms RTB

RTB, 'nnn' byte record too large for user's buffer

Facility: RMS, OpenVMS Record Management Services

Explanation: A record that was retrieved from a file by an RMS $GET
operation within an application program is too large for the
buffer provided by the application. The status value (STV)
field of the RAB contains the size of the record that is too
large; the returned record is truncated to the size of the
user buffer.

User Action: Ensure that the RMS record structure and the RMS file
attributes on the associated RMS file meet the requirements
of the application. If they do, the application program must
be rebuilt or reconfigured to provide a buffer large enough to
read the failing record from the file. alternatively, the size
of the failing record must be reduced to fit into the buffer
provided by the application.

Re: 11382 byte record too large for user's buffer

Robert Gezelter — Mon, 13 Jun 2011 15:34:49 GMT

Andrew,

Ok, first let us do the science.

I strongly suggest doing a DUMP/OUTPUT=/HEADER on the file. Then take a look at precisely what RMS has for the file format (and whether indeed the contents are correct in the context of that information).

DCL has a hardwired limit, as do many of the utilities (DCL symbols are also limited, the precise length depends on which version of OpenVMS is in use).

If the file contains records that are too long, then a simple program in any one of the normal programming languages (e.g., FORTRAN, C/C++, COBOL, BASIC, ...) will be able to read the file without problem.

Further discussion would be helped by seeing the header information displayed by DUMP.

- Bob Gezelter, http://www.rlgsc.com

Re: 11382 byte record too large for user's buffer

RBrown_1 — Mon, 13 Jun 2011 19:17:57 GMT

I most often see this when I get a text file from a unix-like system. VMS creates the file with the usual variable length records, but the data is still stream or stream-lf. Or worse, stream-lf embedded in a single humongo variable length record.

With my ancient version of VMS and UCX, I have my best luck with transferring text files from unix-like OS in binary mode ftp and then SET FILE/ATTRIBUTES=RFM=STMLF.

To troubleshoot this:

1. DIRECTORY/FULL filename;0

What is the record format?

2. DUMP/BLOCK=COUNT=1 filename

Does it look like the record format found in step 1? If not, what does it look like?

? Does each line start with a 1-word byte oount? Variable length records
? Does each line end with ? Stream
? Does each line end with ? Stream-LF
? Does each line end with ? Stream-CR

3. HELP SET FILE/ATTRIBUTES

4. SET FILE/ATTRIBUTES=RFM:xxx

Re: 11382 byte record too large for user's buffer

John McL — Mon, 13 Jun 2011 21:59:34 GMT

1 - Save the original file and make one or more copies to experiment with.

2 - Do DIR/FULL and look particularly at "record attributes" and "record format". To get the best help with this problem post the full DIR/FULL output here (as a text attachment)

3 - I'm guessing that you have a stream-LF file but VMS has the flags set to indicate some other file type. To change the setting, use the command
$ SET FILE/ATTRIBUTE=(RFM:STMLF)

Do that on the copy of the file to test it because my guess might not be correct.

I repeat ... posting the output of DIR/FULL here (as a text attachment) will get you the best solution.

Re: 11382 byte record too large for user's buffer

Robert Gezelter — Tue, 14 Jun 2011 09:52:25 GMT

Andrew,

DUMP/HEX/HEADER/BLOCK=(START:0,END:45) is far preferable to DIRECTORY/FULL. One needs to know BOTH what RMS thinks the attributes are as well as the actual file contents to see whether the RMS attributes are set correctly or incorrectly.

- Bob Gezelter, http://www.rlgsc.com

Re: 11382 byte record too large for user's buffer

A.W.R — Wed, 15 Jun 2011 08:53:01 GMT

Hi,

Thank you for all your input. We found that if we set the file attributes:-

set file/attr=(rfm:fix,mrs:512,lrl=512,org=seq,rat=none) 'p1

On the file in argument 'p1. It changes the record format, and we can open the file. We can then scan the file and parse out the data we need.

Kind regards
Andrew

Re: 11382 byte record too large for user's buffer

Hein van den Heuvel — Mon, 08 Aug 2011 20:23:29 GMT

I'm glad the problem is solved, but I'm a little disappointed with the replies so far.

Nobody explained the 'type' problem, which is simply due to SYSGEN param MAXBUF being too small.

http://h30499.www3.hp.com/t5/System-Management/Quota-Problems/m-p/4087165#M18980

Cheers,
Hein