Usable memory limit for Linux on HP dc7600?

[andyt22]

I have recently upgraded the memory of some HP dc7600 SFF desktop PCs to 4 Gb by replacing the original 2 x 512 Mb DIMMs with 4 x 1 Gb DIMMs. The BIOS reports 4 Gb of installed memory but if the BIOS memory test utility is executed, it will count up to 3346 Mb and then suddenly jumps to 4096 Mb before the test ends.

When the systems boot into 32-bit Linux, the maximum memory available as reported by free or /proc/meminfo is 3347 Mb when running Ubuntu Server 10.04 and 3340 Mb when running Red Hat Enterprise Linux Server 5.7 But on running dmidecode, it correctly finds 4 x 1 Gb DIMMs fitted. 

Why is part of the top 1 gigabyte of memory not being made available to Linux? These same operating system versions running on different hardware (eg, SuperMicro dual Xeon 32-bit motherboards) mange to see and use all 4 Gb of memory. Have I missed some BIOS setiing? And why does the memory test counter jump from 3346 Mb to 4096Mb?

Andy 

[Matti_Kurkela]

Please see the Technical Reference Guide for dc7600.

On page 53, you'll find the system memory map. Despite the traditional name, it might be better to call it an "address space map" instead of a memory map.

When you are running a 32-bit OS, the system uses 32-bit numbers as memory addresses. Since 32-bit binary numbers are uncomfortably long, they are typically presented in hexadecimal form (base-16). The largest possible 32-bit number is FFFF FFFFh, or 4 294 967 295 in more human-friendly base-10 numbers. This is exactly 4 * 1024 * 1024 * 1024.

In other words, a 32-bit OS has an address space of exactly 4 (binary) gigabytes - so it all should fit in. But the system must fit other things besides RAM memory into its address space. According to the above-mentioned memory map, the top region of the address space is occupied by at least three things that are not regular memory: the High BIOS area, the DMI/APIC area and the PCI memory area. The regular system RAM begins at address 0000 0000h at the bottom and extends to the level marked "Top of DRAM". The non-RAM things in the DOS Compatibility area can usually be switched off or moved elsewhere when the system is not running DOS.

The size of the two former ones is specified: the DMI/APIC area and the high BIOS together occupy the address space between F000 0000h and FFFF FFFFh (inclusive). That is 1000 0000h =  268 435 456 bytes, or 268 megabytes of address space that might not be available as regular memory. Then there is the PCI memory area, whose size is unspecified (probably because it might vary depending on whether or not you have PCI/PCIe cards installed. In total, these three blocks might account for most or all of the missing memory you're seeing.

However, there is an address space expansion trick that might allow your system to have a full 4 GB of memory and still have access to all the necessary non-memory blocks. It is called Physical Address Extension, or PAE for short.

With Linux, you can use PAE simply by choosing to run a PAE-enabled kernel. If you're running RHEL, just install the kernel-PAE package and reboot to start using the PAE kernel instead of the regular one. PAE is not enabled by default because it causes some overhead: if you don't need PAE trickery, the non-PAE kernel is slightly faster.

Instructions for installing the PAE kernel on RHEL

PAE is an acceptable solution for usefully getting 4 GB or somewhat more RAM in a 32-bit system. But if you know you will need significantly more than 4 GB, you should be looking at 64-bit OSs instead of trying to cope with PAE, unless you have a strong reason to stay with a 32-bit OS. The 64-bit versions of both Linux and Windows will happily run 32-bit application software, as long as 32-bit versions of the system libraries are available.