ARIES Problem Executing Fortran Code?

[jakolodziej]

We have two rx2800 Integrity Servers running HP-UX11i v3 with March 2014 patches. This equipment replaces three HP9000 Servers running HP-UX 10.20 (we actually have three identical Systems - 6 Integrity Servers replacing 9 K-Servers). There are about 50 HP Visualize C3600 workstations running HP-UX 10.20 supporting the three Systems. A colleague and I have been assigned to get these Systems running. We are almost at the end, and ARIES has done a great job executing our PA-RISC code. There may be, however, one exception.

One of our Applications (which ran on the K-Servers in the old system) runs forever when we invoke it on the Integrity Servers. My colleague made some changes to allow it to run on the C3600 workstations, and it runs normally to completion (the same way it ran on the old K-Servers). This has led my colleague to believe it's a shortcoming with ARIES, but I would like to know for sure.

The code that runs forever is some old Fortran code. My colleague has mentioned that the problem may be that the Fortran code creates a whole bunch of arrays that may cause a problem for ARIES somehow. When I look at the original delivery instructions for this Fortran code, it requires some kernel parameters to be changed. They are: maxssiz, nfile, nflock, and maxfiles. The delivery instructions indicate only the Server kernel parameters need to be changed. My colleague said he changed these parameters on the Integrity Servers, and even made them much bigger than was required in the original delivery instructions. He even went in and created a ".ariesrc" file where he made these parameters much larger than specified in the original delivery instructions. None of his changes allowed the Fortran code to run to completion.

I know my problem description is very vague, as it's based on a brief conversation I had with my colleague. I have not seen the problem first-hand either, but I believe my colleague (he's the local guru). The Fortran code runs to completion on the C3600 (HP-UX 10.20) with the kernel parameters on the workstation changed.So, on the native PA-RISC environment, the Fortran code completes. Under ARIES, it does not. Since ARIES has performed admirably, I am not that familiar with how to debug this problem. Any pointers on things to try to debug this problem?

Regards,

Jeff Kolodziej

P.S. This thread has been moved from General to HP-UX > languages. - Hp Forum Moderator

[Bill Hassell]

...old FORTRAN code...

Not a great description of the real problem, but the issue might be related to some awful coding practices. I once debugged a huge differential equation program for the aerodynamics of flying disk heads. The issue was that each run was taking 2-3 hours but occasionally it would never finish (after 3 days). Not wishing to trace trillions of instructions, I was able to profile the program by logging the execution time in each subroutine. What a surprise to find that 90% of the time was solving the differential equation. What I discovered was coding done by a mathematician rather than a programmer was causing the delays.

The mathematician in writing the convergence code, would multiply the error value times itself. The reason? To normalize the difference to a positive real number. I know...insane for a programmer but logical -- except for two items: two very small numbers less than 1 when multiplied could create an exponent underflow. So in this decimal computer (yes, the Burroughs 3500 was true decimal), the number 10^-51 multiplied times itself gave 10^-102. But the exponent in this computer's math handler was limited to 2 digits and (for FORTRAN code) silently computed the exponent to -02, a much bigger number. So the code would recalculate again into an endless loop.

The second item is that the math expert used a very expensive computation (multiply) to remove the sign.

I changed the code to set the absolute value of the error rather than multiply.

As if by magic, the multi-hour processing changed to less than 10 minutes and never locked up again.

The difference between PARISC and IA64 with Aries may not be a flaw in Aries, but more likely  a difference in handling math issues where finite number representations may create undiagnosed coding errors.

[jakolodziej]

Dear Mr. Hassell:

Thanks for the quick reply. Yes, you're correct - old Fortran code and a very vague problem description on my part. I was looking for general ARIES-specific debugging techniques. Maybe there's a specific switch or flag I can turn on in ARIES to get a clue of what's wrong. Something along those lines...

I have looked at the code some more, and it talks about "tricky I/O" techniques. Here is a part of the program prologue. "This program does some tricky file I/O. It needs to write to a possibly large number of temporary files. HP Fortran only allows 60 open files at a time. ..." The prologue continues into some implementation-specific discussion where it appears there are many more files open than the 60 limit. I am now looking into the code itself to verify what's discussed in the prologue.

Can I instruct ARIES to allow more files to be opened during the running of this program (my colleague did change the "nfiles" kernel parameter)? Again, the code runs in an HP-UX 10.20 environment, but not an HP-UX 11i v3 environment.

Regards,

Jeff Kolodziej

[Bill Hassell]

60 files sounds like an old default limit for maxfiles.

The nfile kernel parameter is total for the entire kernel, whereas maxfiles (and maxfiles_lim) is for a single process. Check the 3 kernel parameters from 10.20. nfile could be 10,000 or 100,000 depending on what all the processes and networking needs at the same time. The maxfiles... parameters are constrained to less than 1 million although lab testing did not go beyond 400k. Any program that is opening more than a few hundred files at the same time is suspect anyways but I realize that this is an inherited program and probably can't be changed easily.

If the program hit the maxfiles limit, it should have aborted with a file open failure. Does the program report any kind of error messages?

[Dennis Handly]

>One of our Applications runs forever

Aries won't help much if these are CPU bound.

What does top say?  100%?

If you expect I/O, you can use tusc to get the system calls.

These will tell you what files are opened and what's begin read/written.

Using lsof can tell you any open files and the file pointer values.

You'll need to get tusc and lsof from the Porting Center.

[Rajesh K Chaurasia]

Hi Jeff:

The problem description does not help troubleshooting. ARIES does not have any problem in supporting open of large number of files (provided nfiles kernel parameter is large enough) or letting emulated PA-RISC application utilize large memory regions (stack, heap, mmap etc).

Floating point code emulation suffers performance overhead under ARIES due to architectural differences between PA-RISC and Itanium

• PA-RISC has delayed floating point exception, while Itanium has immediate FP exception. This means emulated code can only operate on a copy of emulated FP registers mapped to native Itanium registers. It would be semantically incorrect to emulate PA-RISC FP code with immediate exception semantics (emulated signal handlers may get confused). This acutally requires emulated FP context to reside in memory and copied to Itanium FP registers when executing in translated code.
• PA-RISC has FP register half access for left and right 32-bits of 64-bits wide FP register. There is no such provision on Itanium. This also needs emulated FP context to reside in memory.
• Effectively, emulation of FP instructions requires copying of FP operands from memory to Itanium FP registers and copying of result back to memory. So an already costly FP operation becomes even more costlier by way to multiple memory accesses.
• PA-RISC floating point status register (FPSR) does not directly map to Itanium FPSR and is kept in memory and assembled into native FPSR for each translated code block.
• Several sub-opcodes for various floating point operations are executed through exception trap to OS. ARIES emulation needs to bridge the differences in PA-RISC and Itanium implementation.

Please refer to aries(5) man page and find details about an option to map FP context in genral registers (-opt_fpgr) which would eliminate memory accesses from transated PA-RISC fP code. Depending on execution profile of your application, your mileage may vary including minor slowdown.

In a nutshell, one cannot expect very good performance for PA-RISC floating point intensive application on Itanium under ARIES emulation. Only way for better throughput would be reduce the amount of FP code in execution stream.

Make sure you are using latest ARIES patches. You may want to report this issue to HP response center who would advise accordingly on further triaging and troubleshooting help.

Regards

-Rajesh

[jakolodziej]

Dear Mr. Hassell:

Thanks for the quick reply and the advice. I have been off trying to solve the problem, so I am tardy in my reply. I thought I would try to use gpm/glance/lsof to try and debug my problem further. I looked at your suggestions, and found that "nfiles" is deprecated in 11i v3. I also found problems with our Application (pilot-errors on my part) that is preventing me from reproducing the problem. I wound up just executing the program that produces the error, and ARIES core-dumped with an error indicating I should increase "pa_maxssiz_32bit" kernel parameter and increase "-ssz" parameter in .ariesrc.I did both with no luck. My approach now is to get to the point where I can reproduce the problem, and go from there. I will certainly keep changing "maxfiles" in mind. I do not want to waste any more of your time until I can get a better handle on what the problem is and ask better questions.

I have assigned kudos - thanks for the help.

Regards,

Jeff Kolodziej

jeffrey.a.kolodziej@nasa.gov

[jakolodziej]

Dear Mr. Handly:

Thanks for the quick reply to my problem and the help. I suspect my colleague would have told me if the process was cpu-bound. I did not ask him if he used "top", but I will keep it in mind proceeding forward. My current plan is to first reproduce the problem myself - which I am having a problem with. I will work with my colleague to make that happen, and proceed from there. I do not want to waste any more of your time - I need to ask more specific questions.

I have assigned kudos - thanks for the help.

Regards,

Jeff Kolodziej

jeffrey.a.kolodziej@nasa.gov

[jakolodziej]

Dear Mr. Chaurasia:

Thanks for the reply and the info. If I understand your reply, then my problem is not a shortcoming of ARIES. My problem is some option I need that I am not selecting. That's good to know. I'll look at the "-op_fpgr" option. I have spent a fair amount of time lately with the manual page, but have not studied this option.

As for the latest patches, I am in complete agreement. I have mentioned that fact too many times to "the project" lately and am being ignored. The "squeaky wheel" does not always get the grease!

Thanks for the help. I will assign kudos.

Regards,

Jeff Kolodziej

jeffrey.a.kolodziej@nasa.gov