topic Re: DCL command buffer overflow in Operating System - OpenVMS

DCL command buffer overflow

Wim Van den Wyngaert — Mon, 03 Mar 2008 09:56:24 GMT

VMS 7.3

$ Read x x_rec
$ y=f$extr(0,10,x_rec)
DCL-W-BUFOV, command buffer overflow

x is a file with long records (1200 char).

How to get around this overflow ?

Wim

Re: DCL command buffer overflow

Karl Rohwedder — Mon, 03 Mar 2008 10:38:45 GMT

Seems to be on VAX :-)

You may try the EXTRACT (freeware CD) utility, gathering the output via the PIPE command:

$ pip extract sc:a.dat /col=(1:10) | (read sys$pipe a ; def/job test &A)
$ y = F$trnlnm("TEST")

or

write your own program.

regards Kalle

Re: DCL command buffer overflow

Wim Van den Wyngaert — Mon, 03 Mar 2008 10:43:20 GMT

No VAX but Alpha. No way getting around without installing stuff ?

BTW : I don't need the whole record, only the beginning and I prefer not to do a convert to truncate the file.

Wim

Re: DCL command buffer overflow

Volker Halle — Mon, 03 Mar 2008 10:44:47 GMT

Wim,

if you are ONLY interested in the first 10 bytes of each record, CONVERT/FDL=.../TRUNCATE to a temp file first, then process the temp file.

Or upgrade to V7.3-2 or higher to be able to use Extended DCL - but you know that ;-)

Volker.

Re: DCL command buffer overflow

Wim Van den Wyngaert — Mon, 03 Mar 2008 10:48:55 GMT

I tried this but no error and not working. Funny bug or feature.

>open/read x login.com
>read x xx
>sh symb xx
XX = "$ set nover"
>read x xx[0,10]
>sh symb xx
XX = "$ set nover"

(and there is only ser verify)

Wim

Re: DCL command buffer overflow

Robert Gezelter — Mon, 03 Mar 2008 11:11:50 GMT

Wim,

I do not believe that partial assignments to strings were supported in the DCL READ command (the HELP text is specific--"symbol-name").

The lack of a reported syntax error is certainly a bug (although I do not have the time to verify it, or its age).

- Bob Gezelter, http://www.rlgsc.com

Re: DCL command buffer overflow

John Gillings — Mon, 03 Mar 2008 20:42:11 GMT

Wim,

(sigh), Yes, DCL has limitations. That particular limitation was extended several years ago (>5) in EDCL, available in V7.3-2 and above (why are you still on V7.3? -2 is guaranteed to be binary compatible with all non-privileged code - it's a trivial upgrade, well tested and very well understood). Sorry, there are no valid excuses for running V7.3. I'll accept there are some arguments for V6.2 and V5.5-2, but NOT V7.3.

If you're not prepared to install a later version, then use a different programming language that doesn't have the same limitations. Even if you don't have any other compilers, for the specific issue here, (copy the first 10 characters of the first record in a file into a specific symbol), you could write a less than 20 line MACRO32 program which would work on any OpenVMS system, any version, any platform.

Re: DCL command buffer overflow

Wim Van den Wyngaert — Tue, 04 Mar 2008 07:11:08 GMT

John,

This is a bank and lots of money flows thru the VMS platform. No way to upgrade (already on lots of unsupported stuff of Sybase and Reuters, core people gone with the wind) unless you sign a document that you will pay for all the damages. But bring a large check in that case. And a very rich sponsor.

BTW : it's 7.3 not fully patched.

Wim

Re: DCL command buffer overflow

Ian Miller. — Tue, 04 Mar 2008 16:34:22 GMT

*insert usual stuff about risks of running unsupported versions versus risk of upgrading*

Follow the advice of JG and use another language. The version of DCL you are running has limits and you have hit one of them.

Re: DCL command buffer overflow

Wim Van den Wyngaert — Tue, 04 Mar 2008 16:43:51 GMT

I went for the convert of the file before using it.

Re: DCL command buffer overflow

John Gillings — Tue, 04 Mar 2008 20:39:02 GMT

Wim,

That's a load of *&^%#*($

One would have expected that "a bank and lots of money flows thru" would have a much better appreciation risk management than demonstrated by insistence on remaining on an old, unsupported, and unsupportable version of an operating system, with numerous KNOWN problems.

How will they ever upgrade? What will they do if they hit one of the many bugs which have long since been fixed by patches that are tried and tested across the planet? How will anyone even diagnose problems on a system with unique version and patch levels? Don't they have test systems? What if they have a need for a new or fixed feature? (for example, longer command line buffers).

It seems obvious to me that the risk they think they're avoiding is FAR FAR less than the risks they expose themselves to by NOT upgrading.

I could never understand why some users would refuse to upgrade a supported, QA tested minor version, but would happily install and run with a hacked up, untested, unique patch when they hit a problem.

They same bank no doubt blindly downloads and installs all the patches that another software company throws at them...

This is a huge issue in the OpenVMS world. Far too many versions, and far too many people unnecessarily afraid of the upgrade bogeyman. It's up to professionals like YOU to educate those that make the decisions.

OpenVMS should NOT be fragmented into a zillion incremental versions. It makes everyone's life harder and GENERATES risk, rather than mitigating it.

Re: DCL command buffer overflow

Jon Pinkley — Tue, 04 Mar 2008 23:23:51 GMT

John,

Although I agree with you about running old versions, if you were at that site, and given the same restrictions, specifically, "guarantee that this upgrade won't break anything, or pay for all the damages", your only choices are to leave it alone or find another client.

Yes, you can try to educate them, and point out that the risks associated with old versions may be higher that the risk of upgrade. However, if something breaks because of a deficiency of the old system, that can't be blamed on you. Any problems that occur after the upgrade will be blamed on you whether they are in fact related to the upgrade or not.

"core people gone with the wind", this is another problem. What if there is something that breaks as the result of a change? And there is no documentation? Hopefully the those problems would show up before going live, but you often can't do a 100% test on a test system, and with a banking system, I would expect a lot of interaction with outside systems that would be hard to faithfully emulate.

They should definitely have a test system, but doing a good regression test/parallel testing, etc. is not a trivial matter. Although I would be interested in ideas about how those can be done with good coverage and minimal human resources.

I remember when I was in high school (~1973) we went to a NORAD (Air Defense) monitoring site as a school field trip for our Physics class. They were still using vacuum tube computers at that time (possibly due to reduced susceptibility to EMP). So there are some industries/markets that are extremely lethargic in their upgrade schedules. And they may not upgrade until it is a absolute requirement because of non-support of the replacement platform.

Jon

Re: DCL command buffer overflow

Wim Van den Wyngaert — Wed, 05 Mar 2008 07:22:56 GMT

Jon(h),

Sorry but can't give more details but it's worse than you think. Officially we are in the last year of VMS ...

Wim

Re: DCL command buffer overflow

Jan van den Ende — Wed, 05 Mar 2008 09:12:14 GMT

Wim,

>>>
Officially we are in the last year of VMS ...
<<<

And THAT at a bank that does not even dare to upgrade???

Are you at liberty to tell us what will be the replacement platform those "... lots of money flows thru" ?
Try to (let them!) explain how that "minor upgrade" will be less risky than the "giant move" of VMS v7.3 to v7.3-2, or, even better, v8.3 !!
And in the course of that, let them explain how they intend to be more trustworthy on the new platform, because somehow I doubt that the replacement would be NSK...

Then again, probably all caused by the lack of (visual) support for VMS by the current owners... Beuh.

just rambling....

Proost.

Have one on me.

jpe

Re: DCL command buffer overflow

Wim Van den Wyngaert — Wed, 05 Mar 2008 09:36:13 GMT

Jan

Most of the stuff is going to Sun. It concerns big trades and currency swaps. They will just do them in other systems (package based, need a lot more Sun hardware than VMS hardware).

But the last 20% will be tricky ...

Wim

Re: DCL command buffer overflow

John Gillings — Wed, 05 Mar 2008 20:17:53 GMT

Wim,

> Officially we are in the last year of VMS ...

Don't worry, I know plenty of people who've been in "the last year of VMS..." perpetually since 1990! ;-)

Re: DCL command buffer overflow

Aaron Sakovich — Wed, 05 Mar 2008 22:12:59 GMT

SOAPBOX:

It never ceases to amaze me at the haughty, condescending attitudes exhibited by some when a person comes here for help, and receives derision for being on an old version of VMS.

Folks, if the user is running some obsolete version of the OS or an app, tell him why his problem exists, suggest he upgrade because that problem was fixed in a later rev, but don't flame him! Some people are overwhelmed as it is, don't need scorn poured on them, and we will never garner converts to VMS if we ridicule people who come here for help!

Geezzzz.... (off my soapbox now...)

Re: DCL command buffer overflow

Jon Pinkley — Thu, 06 Mar 2008 02:14:06 GMT

Wim,

No guarantees this will work, but it does on 7.3-2 with a 8172 length string.

$ read x x_rec
$ y=f$fao("!10AS",x_rec) ! same effect as f$extr(0,10,x_rec) but may work places f$extract fails.

Can you see if that works on your version?

Jon

P.S. here's what I did on 7.3-2 (patched)

$ abc=f$fao("!8173*a")
%DCL-W-BUFOVF, command buffer overflow - shorten expression or command line
$ abc = f$fao("!8172*a") ! 8172 "a" characters 8173 gets buffer overflow
$ write/symbol sys$output abc
%RMS-F-SYS, QIO system service request failed
-SYSTEM-F-EXQUOTA, process quota exceeded
$ new=""
$ new=f$fao("!10AS",abc)
$ sho sym new
NEW = "aaaaaaaaaa"
$
$ write/symbol sys$output abc

Re: DCL command buffer overflow

Karl Rohwedder — Thu, 06 Mar 2008 06:06:32 GMT

Joe,

nice idea with F$Fao, helps on VAX:

$ lang=F$Fao("!1000*A")
$ kurz=F$Extract(0,10,Lang)
%DCL-W-BUFOVF, command buffer overflow - shorten expression or command line
$ kurz=F$Fao("!10AS",lang)
$ sh sym kurz
KURZ = "AAAAAAAAAA"

regards Kalle

Re: DCL command buffer overflow

Jess Goodman — Thu, 06 Mar 2008 18:11:33 GMT

In VMS 7.3 the DOCUMENTED size limit for a DCL symbol was 1024 bytes. With 7.3-2 and Extended DCL this was increased to 8192 bytes.

However a problem wither DCL or EDCL is that when using symbols that are close to these documented limits with most DCL commands you will get the dreaded:
%DCL-W-BUFOVF, command buffer overflow

For instance under VMS 7.3-2:
$ x_rec=f$fao("!8192*a")
%DCL-W-BUFOVF, command buffer overflow
$ x=f$fa("!8175*a") !longest that works
$ write sys$output f$len(x)
%DCL-W-BUFOVF, command buffer overflow
$ say := write sys$output
$ say "''f$len(x)'"
8175

So minor changes in the DCL commands you use can prevent or cause BUFOVF when you are *CLOSE* to the limit. Jon's choice of F$FAO instead of F$EXTRACT is another example of this.

READ and WRITE/SYMBOL work up to and even over the documented limit, but there's no way under VMS 7.3 to do anything else with a symbol size of more than 1024 bytes.

Another limitation under V7.3-2 is LIB$SET_SYMBOL and LIB$GET_SYMBOL only work for symbols up to 4096 bytes.