RH 5.5 32 bit and number / size of java engines.

[TwoProc]

Question: on a RH 5.5 32 bit OS system, how much memory can I address? I know that the whole PAE mode is gone in 5.X and the system just uses PAE normally. So, what does that give me? Usable 32GB ? 64? I know that a 32 bit address range is only 4GB, and that PAE eliminates the 4GB boundary and lets you go further. How much? a few bits more? 32 bits more?

Also, if anyone has experience in this, how far "up" in memory can I run 32 bit java engines? Can I keep making 32 java engines and putting them out in memory all the way to 48GB total memory?

Reason I'm concerned is that I don't want to just assume this is all feasible and suddenly hit an "oh, it doesn't do that past xxxx" number for that type of application... after spending all kinds of time and $$$ planning on a particular layout to work. I'm worried about something goofy like we had/have with HPUX and "memory windows" to make things work, or worse not have it workable beyond some limit at all.

[Lucifer Megacruel]

Hello TwoProc,

With PAE enabled you can address upto 64 Gb theoretically. But the application need to be PAE aware ( I know only theoretical side of it ). In windows you have a smilar concept called Address Windowing Extensions. On a nix box, you might have to make use of mmap() to map regions into a file. You can possibly get help from more experienced members on this issue.

--Lucifer

[Andrew C Fieldsend]

PAE only affects physical memory support; it allows the OS to use the full physical memory of the system if it has more then 4GiB (up to the RH 5.5 limit of 1TiB). The virtual address space for each process is still limited to the 4GiB addressable via 32 bits.

To answer the original question, yes you can keep creating additional 32-bit Java engines until you run out of memory (physical and swap). Each will get its own private 32-bit address space.

[Matti_Kurkela · Accepted solution]

The technical upper limit for PAE is 64 GB, but running a 32-bit system with PAE has some other strict limitations.

First, in Linux, the 32-bit memory address space limitation is still in effect: no process can see more than 4 GB of address space. However, it doesn't have to be the _same_ 4 GB for all processes: each process can see different parts of the larger memory space. (Linux and Windows use PAE in very different ways.)

Second, there's a kernel configuration option called "vmsplit". It dictates how much of the 4 GB address space is reserved for kernel use. Typically, in modern distributions configured for PAE, the vmsplit is set to 3GB/1GB: 3 GB for the user process, 1 GB for the kernel. (You can change this by compiling a custom kernel, if you have special requirements.)

The kernel-reserved 1 GB will always stay mapped in the processor's address space, because it includes page tables (essential for memory management), memory-mapped hardware control registers and kernel-userspace API. So, typically the lowest 1 GB of system memory will be known as "low memory": the rest will be "high memory".

If your hardware includes IOMMU, it can do I/O operations directly to high memory. But many systems with 64-bit-capable Intel processors don't have an IOMMU: these must rely on SWIOTLB, also known as "bounce buffers". When a disk controller needs to deliver some data to a process that does not happen to be in the 32-bit addressable 4 GB memory space at the moment, the data must be passed into a "bounce buffer" located in the low memory, and transferred from there to its ultimate destination when the target process is mapped into the address space again. This causes some overhead in I/O operations.

This also means the low memory becomes a critical resource: it is possible to run out of low memory while still having plenty of high memory free. This is one of the reasons why RedHat recommends using a real 64-bit system instead of PAE whenever possible: RedHat won't even provide support for 32-bit PAE systems with more than 16 GB of memory on RHEL 6, although running such a system may still be technically possible.

Together, the vmsplit and the address space limit work like this: with a 48 GB system, you could run about 15 Java processes of 3 GB each... but even if you had 40 GB of memory free, you could not run even a single process sized 3.1 GB or more.

Please remember that 64-bit x86_64 Linux systems can run 32-bit software just fine.

My opinion: PAE is useful if you need to go only slightly above the 32-bit 4 GB limit, maybe even up to 16 GB or so. But for a 48 GB system, 64-bit OS is clearly a better choice.

Having said that, I currently have some production systems that are running 32-bit with PAE and have about 48 GB of memory... but the corresponding development systems are already 64-bit, and the testing + production will migrate to 64-bit with the next major upgrade.

MK

[Andrew C Fieldsend]

Just to clarify my reply, 1TiB is the limit imposed by the RH 5.5 kernel (even on 64-bit systems), while 64GiB is the limit imposed by the hardware implementation of PAE itself, as Matti says.

[TwoProc]

Matti - wow, excellent response. Thanks for taking the time for the great info, it was more than I asked for, and exactly the information to help me out. Thank *you* very much.

[JL Martinez]

I know I'm late, but don't forget to limit the stack size of the processes. The best thing you can do is to put this on the .bash_profile file of the user launching the JVM:

ulimit -s 8192

It will allow you to get closer to the 4GB limit of the java heap of your JVMs.

Regars.

[Alzhy]

Each Java process served by a 32-bit Java "Virtual Machine" follows the basics of a 32 bit process - whether the OS is 32 bit or 64bit. That means - its data, text and stack will not exceed ~ 4GB of memory - just like any 32 bit process.

As to how many JVMs you can run - it depends on your RAM.

We've since moved our apps to 64bit JVMs (JRE) as the 32 bit JVM simply does not cut it any more.

And as Matti has so eloquently explained - RHEL 64bit OS can run 32 bit apps just fine. We have dropped 32 bit installations of RHEL in our continuing Linux adventure - in fact our fledgling "migration/porting environment" - I've reently installed Intel's C Compiler suite that can build/compile both 32 and 64 bit binaries.

[TwoProc]

Thanks for the responses all. Alzhy, I too would like to use 64bit Java, however I can find no evidence that this is supported by Oracle Enterprise ERP Business Suite (generically known as "Oracle Applications") we're currently in the upgrade cycle to 12i.

From I read, only 32 bit apps are supported for Oracle Applications - so are you successfully using 64 bit java engines for Oracle ERP applications?

Thank you!

[Alzhy]

We have Oracle Weblogic 10.3 -- 32bit. Our JDK/JRE has since been updated to 64bit - no issues nor no recoding of Java sources as far as I can recall...

[TwoProc]

Thanks again - one more question then. Are the libraries of code(jars, etc) that you've received from Oracle's Weblogic compiled as 64 bit programs for you, or is this not an issue with engines that executing in a virtual machine be ther 32 bit or 64? Seems like it would matter to me. Of course, I realize you could recompile your own code, but I'm wondering about the seeded code you get from Oracle...

[Alzhy]

TP, this is what I gathered from our environment. We have 32bit WL 10.3 installed -- it is bundled with Sun Java and Jrockit 32bit. But out apps folks do not use the bundled and outdated java versions in their deployments/java code. Instead - we use a common JDK release linked as follows /usr/java/current -> /usr/java/jdk1.6.0_24 (or to whatever is the latest and greatest) - and our installation is always 64bit. And they simply use javac out of this installation to build their java objects.

As far as the bundled jars (in lib, demo and plugins, etc) with WL10.3 32 bit -- I really could not tell. From what little I know, the jars are archived bits of java objects -- I do not know how to determine bitness of them objects. Maybe use "jar" to extract a few? Again -- I'm no weblogic infrastructure expert -- just relaying what my clients are telling me but I observe though - the running Java proces out of our WL10.3 servers are 64bit JVMs.

[Andrew C Fieldsend]

All compiled Java code is platform-independent, unless it links to JNI libraries, which are necessarily platform-specific.

The Java language specification was deliberately written to remove all questions of word length and bit-endianness from consideration so that any Java program would be "write once, run everywhere".

If your WebLogic application is written in pure Java, it will make no difference whether you run it on a 32- or 64-bit JVM (other than the speed and memory usage).