topic Re: HP ProLiant Server ML370 G3 with 12GB RAM in Operating System - Linux

HP ProLiant Server ML370 G3 with 12GB RAM

Yildiz_1 — Thu, 16 Jun 2005 06:32:40 GMT

Hello,
I have an HP ProLiant Server ML370 G3 with 12 GB RAM running as a database server (Oracle 8i) having Debian 3.0r1 installed with kernel 2.4.26 #2 SMP.
2 x Xeon 2,8 GHz
1 HP Smart Array 641
1 HP Smart Array 5312
HP Platten Array SW4354 with cciss 2.4.50 driver.
With 12 GB RAM the system hangs several times during the day for some minutes (2-25 minutes). If I remove 8 GB RAM and have only 4 Gb the system works normal.
Does anyone had this problem? What could cause the system hang?

thanks for any comment.

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Stuart Browne — Thu, 16 Jun 2005 06:38:22 GMT

Start by finding out what kernel config options were used.

In particular, you need to verify that the 'CONFIG_HIGHMEN64G' option is set to 'Y'.

You didn't state whether the OS was seeing all 12GB of the RAM when it was in the system, or only 4GB.

As for what could cause hte system to hang like that is a bit up in the air however.

What are the load averages before it hangs? Do existing login sessions continue to work or do they also hang? Can you ping the machine or get any other sort of network activity during the hang?

What would also be nice is to get a feel for the sort of disk IO that's going on when these hangs occur. Use 'vmstat' or 'sar' to get an idea of the IO.

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Yildiz_1 — Thu, 16 Jun 2005 06:58:15 GMT

Hi,
yes we always have 'CONFIG_HIGHMEM64G' Y .
During the hang , it is even not possible to run a command in the shell, therfore it is difficult to look at the system resources. The host is replying to a ping!
I have read something about disabling 64 bit DMA for the cciss driver but this must already been done with kernel 2.4.26 and cciss 2.4.50, I think.

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Stuart Browne — Thu, 16 Jun 2005 07:04:28 GMT

Ok, time for dumb questions.

How much Swap do you have, and what's normal swap utilization like?

The other one is the Oracle tuning of smephores etc.. Are they all done according to the Oracle documentation?

Beyond this, I'm grasping at straws. I've not had hands-on experience with Oracle in such situations, sorry. Wait for SEP to wake up, hopefully he'll be a bit more helpful.

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Yildiz_1 — Thu, 16 Jun 2005 07:12:35 GMT

Hi,
sysctl.conf:
kernel.shmmax = 4294967290
kernel.sem = 1024 32000 128 512
free:
total used free shared buffers cached
Mem: 3882992 3775320 107672 0 16916 2984552
-/+ buffers/cache: 773852 3109140
Swap: 2097136 86672 2010464

Oracle technical support comment on this problem :
Oracle (like any other piece of software), does
not ask for specific memory , it ask for a chunk of X size and the OS gives it
to it (all we know of is the virtual address of that memory). You will solve th
is problem alot faster if you consult the people who can offer the best diagnost
ic software - and , as this is a problem that effects the whole OS, then those
people are the OS vendors.

Has the OS vendor analyzed this ?

I can only hope that someone can help me!

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Gopi Sekar — Thu, 16 Jun 2005 07:58:05 GMT

Since you are saying if you remove 8GB RAM it works fine.it would be better if you run memory diagnostics on the server. Download memtest86 program (or it is available as part of SUSE install boot menu on CD1) run full diagnostics and see whether it reports any problems.

Debian should also contain this package, if so take from there.

Hope this helps,
Gopi

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Ross Minkov — Thu, 16 Jun 2005 08:02:37 GMT

I'd run the memory tests from the ProLiant CD to eliminate any possibility of memory problems first. Since you are running Debian, I don't think that you can download the ProLiant Support Pack for Linux. I think that the PSP is only available for RH & SuSE, but you can check on hp.com.

-Ross

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Yildiz_1 — Fri, 17 Jun 2005 09:00:55 GMT

Hi,
we testet the RAM with memtest and some other tools , no errors found.
I think the best to do is installing a certified distribution of Linux so that I can everything push to the Hardware vendor or oracle, isn't it?
With Debain I have to find out the problem by myself even if it is a failure of hardware, driver or a kernel bug!

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Gopi Sekar — Fri, 17 Jun 2005 09:27:03 GMT

yeah, next thing what I would suggest is to install a latest OS on it and test the reliability. it could be problem with the debian kernel too.

Hope this helps,
Gopi

Re: HP ProLiant Server ML370 G3 with 12GB RAM

xyko_1 — Fri, 17 Jun 2005 14:37:33 GMT

Hi Kinzinger,

when you took the memory picture of your system

free:
total used free shared buffers cached
Mem: 3882992 3775320 107672 0 16916 2984552
-/+ buffers/cache: 773852 3109140
Swap: 2097136 86672 2010464

the system had 12Gb or 4Gb installed ?

free command shows only 4 Gb total.

regards,
Xyko

Re: HP ProLiant Server ML370 G3 with 12GB RAM

Yildiz_1 — Mon, 20 Jun 2005 07:10:14 GMT

sorry,
the free output is with 4 GB RAM!