Overflow

[Jan van den Ende]

Getting philosofical...

The last few weeks several times the issue of overflow on (especially ported) software has come up several times.
The issue kept nagging, and this afternoon a memory from way back when popped up.

In a former life I did (mostly Filetab and some) Cobol programming on ICL 1900-series "mainframes".

The specific issue that flashed back from the past was a "Special Steering Command" in ICL's Cobol: "WITH BOUNDS CHECKED".
It caused the compiler to generate extra code, which on any data transfer statement checked for Numeric values NOT overflowing/underflowing the defined domains (specifiable, default the full range of the PIC);
and for strings for not TRYING to move larger-than-intended.
Originally this option's main intention was to use it during program development & testing; and when the program went to production the option was removed for performance reasons.

Point is: any data over- or underflowing transfer would halt the program, and generate output that held sufficient info to analyse the issue.

From a VMS perspective such instructions are much better visualised as compiler qualifiers, but the intent should be clear.


Now I am gonna profile myself as a C illiterate, but:

WOULD IT BE TOTALLY INCONCEIVABLE to have this built into the C (C++, others?) compiler?

I imagine that the bulk of VMS sites would gladly accept the performance drawback to get the resulting protection, and the immediate warning upon attempted assault, when using ported software.


Any comments as to desirability, feasability, whatever other issue???


(For my Belgian friends: I had 3 Duvels before writing this, could that be what made me philosofical, or...?)

Cheers.

Join me in my next Duvel.

Jan

[Ian Miller.]

Is that three Duvel Tripels ! (10% by vol for those not following every detail).

Parhaps you could post a message here
http://h71000.www7.hp.com/commercial/c/c_feedback.html

There is the /IEEE_MODE qualifier
http://h71000.www7.hp.com/commercial/c/docs/5492p004.html#index_x_113
/IEEE_MODE=INEXACT may be what you want.

[Ian Miller.]

I've read a pile of stuff of on floating point on itanium and don't remember what checks are available.

Having the option for extra checking would be a good thing.

[Jan van den Ende]

Ian,

Well, Duvel is not fully 10%. (8.5, and by now I finished the 4th as well)
And is is NOT Tripel (although, if you want to compare it, probably triple comes closest).
The name "Duvel" is dialectic for "Devil", and the story is that it points back to a remark by one of the original brewers: the beer behaved so unpredictable in pooring, drinking, and storing, (and morning-after)that it was "develish". The name stuck, and became the brand.


To (try to) become serious: if this thread will (hopefully) generate support for the idea, I think I will have to enter the formal request.

Your second pointer: this is about what I had expected to find, not too much unlike what I know from languages I know somewhat better.
The available checking is on NUMERIC values.
What made me start this discussion was the availability OVER TWENTY YEARS AGO, of an overflow test FOR STRINGS!

Writing all this down, I also started wondering how much of this is already covered by Hoff's stack-execution-protection in IA64 (which MIGHT even be backported to Alpha).

Maybe someone much deeper into compilers and/or internal workings can shine a light on this?

(and now my wife really is wanting me to quit this, so GOODNIGHT)

Cheers.

Have one on me.

Jan

[John Gillings]

Jan,

Compilers have come a long way since the days of mainframes that did little or no checks on calculations. All VAX and Alpha compilers have had range, bounds, overflow and underflow checks, either by default or as an option. One of the "new" features of the VAX architecture back 25 years ago was useable under and over flow checking in hardware for all data types. VAX programmers have always been agast when they learn that some systems did NOT crash a program for a divide by zero error or similar arithmetic fault.

Ian's point is that IEEE floating point arithmetic has even more options for handing over and underflow conditions than VAX had. If C and C++ there is no packed decimal format, see the IEEE related qualifiers to select the most appropriate for your needs.

For COBOL packed decimal arithmetic, these days it's mostly implemented by converting to 64 bit INTEGER for the calculations and converting back to store. That's because 64 bit arithmetic has sufficient precision. COBOL also implements /CHECK=DECIMAL which does bounds checking, as well as "sanity" checking.

Pascal has /CHECK=OVERFLOW.
Fortran has /CHECK=(OVERFLOW,FP_EXCEPTIONS,UNDERFLOW)

Please provide an example of a program with the kind of error that you're thinking of where you don't get useful errors by default, or cannot request errors with an existing compiler qualifier.

[Jan van den Ende]

John,

what I was specifically thinking of was the fenomenon in ported C-language programs where buffer overflow capability generates security holes.

And although in my old ICL Cobol days THAT was not an issue, it was possible for code WITHOUT the BOUNDS CHECKED to mis-manipulate strings to fill address space beyond the areaintended for the string.
The simple demo program added 1 character to a growing string in a loop, that for the demo purpose ran much longer than the receiving string could handle.
WITH the bounds checked, the programm AB-ENDED, indicating why and when it broke.
WITHOUT the bounds checked, the programm went trough the loop much too often, and corrupted the datafile that it was updating.
With a minor input change it ran still longer, and then crashed because some instructions had been changed into something invalid,and were attempted to be executed. In that case the post-mortum was also of little value.

So: would it be somehow possible to have the option to have the compiler generate code that would detect the infamous string (buffer-) overflows, and tell the user WHY the code is not functioning.

-- and I really have not the faintest idea how this should be implemented. Maybe by simply implementing the null-terminated string as a descriptor-described entity that happens to have a null as last byte.
(this may be utter nonsense though, just my shot in the dark).

Cheers.

Have one on me.

Jan

[John Gillings]

Jan,

Sorry, I misunderstood which type of overflow you were talking about.

All VAX and Alpha compilers have /CHECK=BOUNDS which will generate code to check the array bounds of all stores. However, this unfortunately won't always help C and C++.

The issue mostly has to do with the way strings are handled - length being defined by null termination. In most other languages, there is a mechanism to pass decalred string lengths to subroutines (OpenVMS uses descriptors), but many C and C++ routines are passed only the address of the string. The code assumes (hopes?) that the allocation is "large enough", but has no way of knowing.

C programmers could write code that uses a more complex object to represent strings -one that includes bounds information. There are some newer RTL routines which help. For example, instead of "strcpy" use "strncpy" which includes a maximum length. It's all a matter of defensive programming.

In theory, one could put guard areas around selected objects. For large objects allocate a page on each side and protect it against access (OpenVMS memory protection granularity is one page). This would be expensive, especially for many objects, and really doesn't completely guarantee to trap all overflows (consider, what if the offset from the start of the string is greater than a page? You could leapfrog the string and the guard page and corrupt something else, undetected).

Less expensive in memory, but more expensive in processing is to put smaller guard areas with a "poison pattern". After any write access, check the guard areas to see if the pattern has changed. Again, costly without a guarantee.

Stronger typed languages, like Ada and Pascal will give you much of this protection automatically.

My fear is we could implement more complex and costly "protection" mechanisms in compilers, and the nett result would be programmers just get sloppier and less disciplined! They expect the compiler to catch their mistakes, so don't bother to engineer code properly.

That's the ultimate defense against this kind of error - good sound engineering practices!

[Wim Van den Wyngaert]

Overflow is a serious problem, for Duvels.
When you are to fast, overflow may be a problem. So slow putting into the glas is advised.

For non-Duvel users (called novices) : http://www.duvel.be/pages/Main.aspx?culture=en&pageid=merken/duvel/introductie

I used to be a Cobol expert too : 1 advise : compile your sources on al platforms (Unix, HP3000, VMS) and all compilers will find some bugs. At run time they all check different things. HP3000 had a stack that resulted in run time errors and all compilers had options that checked things in a different way.

When moving a big applic from HP3000 to VMS a lot of errors where found !

Wim

[Antoniov.]

Jan,
your question has thousand answer ...
About outbound trouble, I remember when teached me about trap in if/loop statement.
For example this code may fail:
01 I pic 9(3) comp.
01 A pic X occurs 10 times.
move 1 to I.
perform until (A(I) = "T" or I > 10)
...
add 1 to I;
end-perform.

This code works fine:
01 I pic 9(3) comp.
01 A pic X occurs 10 times.
move 1 to I.
perform until (I > 10 or A(I) = "T")
...
add 1 to I;
end-perform.

In first case, A(I) when I = 11 fail if program compiled with outbound check, works if compiled without check :-O

Antonio Vigliotti

[Ian Miller.]

If the CRTL allocated memory from a zone it created (which I think it does) then that zone could have appropriate options so it could be checked with LIB$VERIFY_VM_ZONE. This could catch overflows of buffers allocated with malloc. You may be able to add your own checking code if the zone used by the CRTL has a obvious name. Use LIB$FIND_VM_ZONE to look through the zones to fine the right one then use LIB$VERIFY_VM_ZONE to check it whenever you like.
I suppose the CRTL could do this (enabled by yet another feature logical).

This would not address overflows of buffers on the stack or in static memory. I suppose the compiler could add extra code for those cases.

C is not a language that supports this sort of checking easily.