Wim,
An "optimization" I've used for a large suite of DCL is to have the main entry point check the location of the files. If they're on a real disk drive, create a RAM disk, move all the procedures to it and reexecute the new copy from the RAM disk. This gave me about 15% speed up, even counting the work to setup and tear down the RAM disk. However, it was pre XFC, so your mileage may vary today.
Another real example... I had a task of parsing some data to gather statistics from several dozen large log files (>1.5GB each). I wrote a DCL prototype and found it was taking about 90 minutes per file. While it was running, I reimplemented the DCL in MACRO32. The MACRO version produced identical output to the DCL, but took less than 60 seconds per file. In this case the end to end runtime of the compiled version was significantly faster than the DCL, including the development, compilation and debugging of the compiled version!
I think sometimes the real potential difference between interpreted code and compiled code is not appreciated. I'd also argue that although DCL is more easily accessible, it's often MUCH harder to write and get it correct than Pascal, FORTRAN, MACRO32 or even C and Basic. It's a fallacy to think that "anyone can do it" when there are so many subtle pitfalls like the myriad of potential single character typos to break programs, and there's no compiler to protect you. Sure, it's great to be able to knock out a page or so of script to run once or twice, but when you're getting into thousands of lines of code, or something that needs frequent modification, or that runs all day, every day, it's well worth putting in the investement to do it properly in the most appropriate language for the task.
That said, please don't get the wrong idea about my attitude to DCL. I write huge volumes of DCL, including procedures of several thousand lines, and even some that run all day, every day (but are not too performance sensitive). There are many things that can be done very easily in DCL that require lots of code in other languages (simple example, string subtraction!), BUT the DCL version often runs orders of magnitude slower.
On the other hand, a DCL advantage in *favour* of performance is to use multiple pipes in order to automatically exploit multiple CPUs and avoid temporary files. I've got lots of DCL that forms itself into complex trees of pipes, with up to a dozen processes, each doing their own subtask. What I lose in process creation overhead, I often gain many times over in avoiding I/O to and from temporary files, and all the directory & file system overhead.
> It also has the advantage that it doesn't
>create extra processes (I pase output of
>ncl, tcpip, ...).
On the contrary! I think you'll find that this could be a fruitful area of optimization. Get rid of all those temporary files! PIPE the output through SEARCH (compiled code, INSTALLed, and highly optimized by some of the best code cutters in the business, like Guy Peleg). Trim it down as much as possible, then back into your DCL parser. Yes it creates extra processes, but compare the overheads! These days avoiding disk I/O is your biggest bang for buck.
The important thing is to realise when you're running up against the inherent limitations of the language (true regardless of what language you're using), and wasting your time trying to achieve a particular objective, when the real solution is to restructure or rewrite in a more appropriate language.
A crucible of informative mistakes
