HPE Community read-only access December 15, 2018
This is a maintenance upgrade. You will be able to read articles and posts, but not post or reply.
Hours:
Dec 15, 4:00 am to 10:00 am UTC
Dec 14, 10:00 pm CST to Dec 15, 4:00 am CST
Dec 14, 8:00 pm PST to Dec 15, 2:00 am PST
Operating System - OpenVMS
cancel
Showing results for 
Search instead for 
Did you mean: 

Java: JRuby on Alpha is slow; any hope for improvement?

 
SOLVED
Go to solution
Ben Armstrong
Regular Advisor

Java: JRuby on Alpha is slow; any hope for improvement?

For some time we have worked on a port of Ruby 1.8.2[1] for OpenVMS, to
the point where it is possible to successfully deploy cross-platform
applications using Ruby on Rails. However, it is cumbersome to keep up
the port on our own. There remains more work than we've been able to
keep up with in order to allow it to be merged upstream, we're lagging
far behind latest release of the language[2], are missing out on a
number of important features, and have some fairly serious performance
issues with Rails.

Seeking a solution to these problems, we spent the afternoon yesterday
installing and testing JRuby[3] on a small test OpenVMS/Alpha V8.3
system (a DS10/600 with 512M RAM and a RAID mirrored pair of 18G 15krpm
discs on Ultra-3 SCSI) with patches up to Feb. 2008. In theory, it runs
on all platforms that Java runs on, including OpenVMS, and will allow us
to keep up with latest releases of Ruby software without having to
maintain the port of the language ourselves. We were pleased to
discover that it does, in fact, run. However, our test results after
tuning system and process quotas as per doc[4] show Jruby on the fast VM
to be anywhere from 70% slower to 6 times slower than our C ruby port.

Is there hope that we might be able to tune further to the point where it
is practical to run applications on this platform? We had hoped JRuby's
performance on OpenVMS would be equal to or better than our C ruby port.

[1] http://vmsruby.rubyforge.org/

[2] http://www.ruby-lang.org/en/news/2007/03/12/ruby-1-8-6-released/

[3] JRuby is a cross-platform implementation of Ruby written in Java. See http://jruby.codehaus.org/

[4] http://h18012.www1.hp.com/java/documentation/1.4.2/ovms/docs/user_guide.html#processquotas
19 REPLIES
labadie_1
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

Have you read this article in the OpenVMS Technical Journal ?

Java and OpenVMS: Myths and realities

http://h71000.www7.hp.com/openvms/journal/v10/javavms.html

May be you could improve a few things with it ?
Hoff
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

There's potentially no good answer available on native OpenVMS, unfortunately.

Java is slow most everywhere I've worked with it. (So too was a Java predecessor UCSD pCode, but then I'd be dating myself to even mention that.) The general discussion then becomes "how slow", and then whether or not the available performance meets minimal requirements.

As for a few potentially cheap fixes, your AlphaServer DS10 box is potentially short on physical memory; I'd monitor that, as well as paging and related I/O activity when you're running your tests. Do check process quotas and working-sets, too, as Java Looms Large.

There is only one (other) organization around that I'm aware of that has been looking at Ruby on OpenVMS, and this port is (further) behind current. Here's their port:

http://mvb.saic.com/freeware/freewarev70/ruby/

There are suggestions around platform vendors that are targeting Ruby and Rails, but most of the gear I'm dealing with toward that end is not HP gear. There are StorageWorks disks around, but...

gSOAP, ODBC or JDBC, or another approach toward data and application encapsulation might be an option here; remote access and linking over into the OpenVMS environment from your (I assume) web platform. Or slightly perversely, running OpenVMS inside an emulation, and running Ruby on Rails in the host environment.

There exists far more free source than the requisite free time and free help inevitably required to keep it upright and running and current.

Within HP, one of the folks to talk to in this general area has traditionally been John Apps, if you wish to send the discussion in the direction of HP.

If there is commercial and more formal interest in ports of these or other open source packages, there are folks here that might be convinced to port Ruby, Fortress or other recent languages or tools (gs/gv, Xpdf, etc) over to OpenVMS; to supplement the languages and the environments that are available from HP.

Stephen Hoffman
HoffmanLabs LLC
Steven Schweda
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

> As for a few potentially cheap fixes, your
> AlphaServer DS10 box is potentially short
> on physical memory; [...]

I'll say. 512MB? Yikes. Your workload may
(must) be different, but my junk XP1000
systems both have the max of 2GB, and could
use more.
Ben Armstrong
Regular Advisor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

labadie,

Interesting. Some of those suggestions may give us even better results with the C ruby implementation.

Hoff,

We did look at main memory during testing. In MON CLUS on an otherwise idle system while running the tests, memory usage peaked at 55%, so I don't think that's our biggest constraint. The working set and page file quotas are at recommended values now.

The page you linked to contains an earlier version of the same port we inherited from Masamichi Akiyoshi.

As for the other suggestions, here's a bit of background. The Ruby application code development we are doing runs in parallel with a porting effort from VMS -> Windows. To be honest, as much as I love VMS, I'm constantly frustrated in my development efforts by the lack of tools that I have come to expect as standard on other platforms, e.g. Linux. So I want to do as little as possible that keeps us tied to VMS. Points against solutions that have us redesigning our applications for the sake of VMS. Points for solutions that allow us to continue developing apps to be deployed on either platform without modification. That was the big appeal of JRuby.

It was a strange set of circumstances that led to us "owning" the Ruby port to begin with. So long as Masamichi (who worked for HP in Japan) was working on the port, we figured it had a future. When he stopped development, we picked it up because we were already deploying software with it and therefore needed it to go on. While I'd love it if HP picked this up, I have to be realistic about it. So far, we haven't had people flocking to our port to help out, so I doubt if there is sufficient demand to make it commercially viable.

Steven,

Yeah, this is only a test system representing the low end of what we're likely to encounter in the field. Even so, from what I observed above, it doesn't look like that's a limiting factor ... yet. And if it is, it wouldn't be hard for us to add more RAM to see if it helps. But I'm not rushing to do that until I'm sure we have a real problem.
Willem Grooters
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

The memory requirements for a somewhat reasonable Java performance are 1Gb memory, and a processor running on 500Mhz. AT LEAST. Furthermore you should tune your VMS box to behave as a Unix one.
Of course it depends on the size of the application, the number, size and lifespan of objects. Smaller applications run pretty usable on smaller systems, bigger ones may require considerable more.

Add Ruby's requirements and 512 Mb will surely be way too small, causing massive paging (you'll hear your drive grind on each garbage collector activity). (Much) more mamory will surely help. Adding processor power will add performance gains as well.

On tools: have you checked out Distributed netnbeans: Netbeans on Windows or Linux, and a (java based) server on VMS. Usable for Java (duh), C, C++, FORTRAN, COBOL, Pascal, DCL at least. It DOES run on 512 Mb, but the more memory and processor power, the luckier teh server runs.
Willem Grooters
OpenVMS Developer & System Manager
Ben Armstrong
Regular Advisor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

I appreciate that the limited memory on the test system is insufficient for running apps of anything more than trivial complexity. My test script, however, was rather small and simple and as I said, I never observed total system memory usage above 55% in my tests, so that would seem to indicate we would not see enormous gains in performance with a memory upgrade for these tests. I can well imagine I'd see even worse performance if I tried to run, say, Rails itself. But without an assurance that I'm on the path to achieving acceptable results, I'm disinclined to spend the time it would take to go further.

The purpose of the exercise was not to evaluate the performance of the test system, but to run some simple trials to see if throwing more resources (time and/or hardware) at the project would be worthwhile. We believe they were fair trials. While my results from those trials indicate it is not worth investigating further, we will keep an eye on JRuby as the developers are actively working on performance issues and may turn out a release in future that is worth considering.

I also posted this afternoon to comp.lang.ruby, linking back to this discussion, in case the developers are following and have anything else for us to try.
T. Uso
Advisor
Solution

Re: Java: JRuby on Alpha is slow; any hope for improvement?

I have also tested Jruby on OpenVMS and it works (both executing a ruby script with the Jruby interpreter and executing bytecode compiled from a ruby script). And yes, it does not perform well. Why? because the OpenVMS filesystem is very slow and the jvm has to handle a lot of jar archives. If you want to greatly improve the Jruby performance, put Jruby in a DECram and build a shadow set with this DECram and a LD device for the persistence.

I have played with the ruby port from the HP japanese guy several years ago and found many limitations. For example, the interpreter uses environment variables to have access to some libraries and of course OpenVMS does not like it. Most of these limitations disappear with Jruby.
Steven Schweda
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

> We did look at main memory during testing.
> [...] memory usage peaked at 55%, so I
> don't think that's our biggest constraint.

Did you look at the page fault rate, too?

> The working set and page file quotas are
> at recommended values now.

Recommended by ...? It may be easy to keep
memory free, if you don't let anyone use it.

More analysis may be useful. (No bets, of
course.)

SET RMS_DEFAULT might have some effect, too.
Ben Armstrong
Regular Advisor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

I don't know what a reasonable page fault rate is for this app, so I don't know what to look for to know if I need to tune it further.

As for recommended values, as I said in my original post, the doc I am using is in footnote 4.
Ben Armstrong
Regular Advisor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

Very interesting, T. Uso! I thought we were alone in the world in our efforts to make Ruby work on VMS.

So did you try putting JRuby in a DECram yourself? If so, do you have any numbers from that experiment and version of the OS used for testing, or if not could you give a rough estimate what sort of factor speedup I'd be looking at?

I'm interested to hear you played with the C ruby port. I'm not sure what you're referring to when you say environment variables are used to access some libraries. Are you talking about the socket library? That's no longer the case in the current version of the port. Do you remember what other limitations you encountered?

Finally, did you just test JRuby or did you ever put it into production use?
T. Uso
Advisor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

I did not yet try putting JRuby in a DECram. The development cycle of Jruby is very fast with a release candidate each month. So I prefer to wait a stable release before doing in-depth tests. But, I observed that the response times for most operations are 5-10 faster when J2EE applications are in a DECram.

When I played with the C ruby port, the script failed each time it called the extras library. The failure was caused by a wrong path which was defined by an environment variable.

I'm interested by ruby which is a nice language (cleaner semantics and syntax than python). I just need a good interpreter/VM in order to use it seriously on OpenVMS.
Craig A Berry
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

"70% slower to 6 times slower than our C ruby port."

That's not at all surprising and actually somewhat better than what benchmarks on other platforms reveal. From typing "jruby performance comparison" into Google, the first thing that comes up indicates JRuby is on average 8 times slower than C Ruby on Linux:

http://tinyurl.com/36e7zl

Your choices are not particularly easy, but seem clear enough. If performance is important, do whatever it takes to keep C Ruby afloat or move to something that others are keeping afloat.

If (much) slower performance has a chance of being "good enough," then keep an eye on JRuby and see how it develops.

I spend an inordinate amount of my own time trying to keep Perl 5 up and running on VMS, so I feel your pain in terms of what it takes to keep a port maintained. The good news is that (thanks to John Malmberg), Perl 5.10.0 is really quite viable on VMS. If Perl is at all palatable to you (it's usually not to Ruby people), there are rails-like frameworks that can provide the functionality you are looking for.

It is rather a shame, though, that Perl, Python, and Ruby are all solving essentially the same VMS portability problems independently of each other. Advances in dynamic language run-times may eventually improve the situation. For example, the Parrot project

http://www.parrotcode.org

seeks to create a single VM that can host Perl 6, Python, Ruby, and other dynamic languages, though as yet it is a long-running research project with no shipping products and I'm not aware of any ports to OpenVMS as yet.
Martin Vorlaender
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

Craig,

>>>
For example, the Parrot project seeks to create a single VM that can host Perl 6, Python, Ruby, and other dynamic languages, though as yet it is a long-running research project with no shipping products and I'm not aware of any ports to OpenVMS as yet.
<<<

At YAPC::Europe 2006 in Birmingham, Bernd Ulmann ( http://www.vaxman.de/ ), Thomas Kratz (sp?), and myself took a look into porting it (using VMS VAX 7.3 on my notebook's SIMH), and found some quirks which were quite readily incorporated into the sources (thanks to Bernhard Schmalhofer).

But since then, I haven't again tried anything in that direction; no idea whether Bernd or Thomas have.

cu,
Martin
T. Uso
Advisor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

Be cautious about this claim "JRuby is on average 8 times slower than C Ruby on Linux". It depends on (i) the Jruby version and (ii) the executive mode (interpreted or compiled) :
- The Jruby developers are working hard to improve performance and the last version is always faster
- A compiled ruby script (bytecode) runs faster than an interpreted ruby script inside Jruby

I think that Jruby in compiled mode will be soon a lot faster than the C ruby interpreter. Sun wants that the hotspot VM becomes a good executive environment for languages such as ruby (Jruby), python (jython), groovy... It seems to be a strategical choice.
Martin Vorlaender
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

I think I have to put some things straight (I'm getting old, I guess).

At YAPC::Europe 2006 in Birmingham, Bernd pushed me into trying to build Parrot on my SIMH. This didn't get very far as I had problems building Perl.

At last year's YAPC::Europe in Vienna, Bernd and Thomas presented some patches for Parrot's build procedures to get things going on VMS; those were incorporated by Bernhard during a BOF session. It became apparent that Parrot's build is very much make biased, so no-go with MMS.

Ben, sorry for spamming your thread.

cu,
Martin
labadie_1
Honored Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

Craig

I have found another document stating more or less the same thing about the relative performance of Jruby

Performance of scripting languages
http://www.cs.purdue.edu/homes/sbarakat/cs456/Scripting.pdf

I read
JRuby, Jython ~4-8x slower than Ruby, Python

And yes, it is a pity that maintainers of Perl, Python and Ruby on Vms solve the same problems independantly, and waste a lot of time.
T. Uso
Advisor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

JRuby 1.1 has been released (5 april) and will be followed by 1.1.x versions each 3-4 weeks.

Here are some preliminary results of the performance (response times) of JRuby 1.1 on OpenVMS/Itanium.

Monitoring performance of Java applications is not an easy task because of the JIT feature of the JVM. Moreover, JRuby has not the usual behavior of most other Java applications that I use :
- putting JRuby in a DECram as no significative effect on the response time
- Jruby on my Windows XP is not faster than on my OpenVMS/Itanium
Therefore, be cautious with my results and conclusions ;-)

I tried to use Jrat 1.1b1 to monitor the response time for each class. But Jrat crashed the JVM just after the instanciation of the ruby interpreter when the interpreter runs the ruby script. So, I used ruby timers inside ruby scripts which is a less reliable method.

Executing a "toy" ruby script (kind of "hello world") takes approximatively 6 secondes! This value is mainly due to the start of the JVM + the instantiation of the ruby interpreter.

Changing the values of WSLIMIT and WSQUOTA has no noticeable effects on the response time (only the frequency of the GC changes). During the start of the JVM, I observed an high page fault rate with a lot of "demand zero" pages.

The JIT effect is very spectacular. When Jruby runs a script which loads several times the "hello world" script, the first execution of the "hello world" takes 450 ms and the last one 1 ms. I saw the same behavior when I manually run a script several times inside the graphical Java irbconsole.

The compiled mode did not significatively improve the performance ;-(

First conclusions :
- Jruby is not a good solution to replace DCL as a script langage because of the overhead of the JVM start; use python instead.
- Jruby might be a viable execution platform (for RoR applications for example) because the JVM is started only once and because of the JIT benefits.

To be done :
- test Jruby in a more realistic conditions (with "true" ruby scripts)
- test the gem implementation
- try to run a RoR application in the compiled mode with Tomcat (I am a born optimist ;-)
Tom O'Toole
Respected Contributor

Re: Java: JRuby on Alpha is slow; any hope for improvement?


It is asking for trouble to use a language implemented written in Java. Is jruby where the ruby community is headed? If so, I have to say that is just idiotic, and will keep intel hopping just to provide 386 level performance.

That you got only six times worse performance is lucky.

yeaze, software today....
Can you imagine if we used PCs to manage our enterprise systems? ... oops.
T. Uso
Advisor

Re: Java: JRuby on Alpha is slow; any hope for improvement?

C is faster than Java. So a C application will be faster than its Java equivalent... in theory but not always in the real life. For example :

- Quercus which is a PHP5 interpreter writen in Java is faster than mod_PHP which is writen in C

- H2 which is a RDBMS writen in Java outperforms most RDBMS writen in C

There is no final rule in the software area ;-)