cmaeventd: page allocation failure error in Suse 9 update 4

[sacthivel.C]

Hi all,

We are using Suse linux 9 update 4.
For the past few days we found that the performance of the server is very low.
We found the following error in /var/log/messages file
----
cmaeventd: page allocation failure. order:0, mode:0x21

Call Trace:{__alloc_pages+1208} {__get_free_pages+237}
{pci_alloc_consistent+111} {:cciss:cmd_alloc+40}
{:cciss:cciss_ioctl+1927} {:xfs:xfs_itobp+429}
{:xfs:xfs_initialize_vnode+740} {blkdev_ioctl+1748}
{:xfs:xfs_lookup+119} {d_splice_alias+250}
{:xfs:linvfs_lookup+105} {real_lookup+123}
{kobj_lookup+330} {sys_ioctl+1023}
{:cciss:cciss_ioctl32_passthru+229}
{compat_sys_ioctl+265} {sys32_open+231}
{sysenter_do_call+27}
Mem-info:
Node 0 DMA per-cpu:
cpu 0 hot: low 2, high 6, batch 1
cpu 0 cold: low 0, high 2, batch 1
cpu 1 hot: low 2, high 6, batch 1
cpu 1 cold: low 0, high 2, batch 1
cpu 2 hot: low 2, high 6, batch 1
cpu 2 cold: low 0, high 2, batch 1
cpu 3 hot: low 2, high 6, batch 1
cpu 3 cold: low 0, high 2, batch 1
cpu 4 hot: low 2, high 6, batch 1
cpu 4 cold: low 0, high 2, batch 1
cpu 5 hot: low 2, high 6, batch 1
cpu 5 cold: low 0, high 2, batch 1
cpu 6 hot: low 2, high 6, batch 1
cpu 6 cold: low 0, high 2, batch 1
cpu 7 hot: low 2, high 6, batch 1
cpu 7 cold: low 0, high 2, batch 1
cpu 8 hot: low 2, high 6, batch 1
cpu 8 cold: low 0, high 2, batch 1
cpu 9 hot: low 2, high 6, batch 1
cpu 9 cold: low 0, high 2, batch 1
cpu 10 hot: low 2, high 6, batch 1
cpu 10 cold: low 0, high 2, batch 1
cpu 11 hot: low 2, high 6, batch 1
cpu 11 cold: low 0, high 2, batch 1
cpu 12 hot: low 2, high 6, batch 1
cpu 12 cold: low 0, high 2, batch 1
cpu 13 hot: low 2, high 6, batch 1
cpu 13 cold: low 0, high 2, batch 1
cpu 14 hot: low 2, high 6, batch 1
cpu 14 cold: low 0, high 2, batch 1
cpu 15 hot: low 2, high 6, batch 1
cpu 15 cold: low 0, high 2, batch 1
Node 0 Normal per-cpu:
cpu 0 hot: low 62, high 186, batch 31
cpu 0 cold: low 0, high 62, batch 31
cpu 1 hot: low 62, high 186, batch 31
cpu 1 cold: low 0, high 62, batch 31
cpu 2 hot: low 62, high 186, batch 31
cpu 2 cold: low 0, high 62, batch 31
cpu 3 hot: low 62, high 186, batch 31
cpu 3 cold: low 0, high 62, batch 31
cpu 4 hot: low 62, high 186, batch 31
cpu 4 cold: low 0, high 62, batch 31
cpu 5 hot: low 62, high 186, batch 31
cpu 5 cold: low 0, high 62, batch 31
cpu 6 hot: low 62, high 186, batch 31
cpu 6 cold: low 0, high 62, batch 31
cpu 7 hot: low 62, high 186, batch 31
cpu 7 cold: low 0, high 62, batch 31
cpu 8 hot: low 62, high 186, batch 31
cpu 8 cold: low 0, high 62, batch 31
cpu 9 hot: low 62, high 186, batch 31
cpu 9 cold: low 0, high 62, batch 31
cpu 10 hot: low 62, high 186, batch 31
cpu 10 cold: low 0, high 62, batch 31
cpu 11 hot: low 62, high 186, batch 31
cpu 11 cold: low 0, high 62, batch 31
cpu 12 hot: low 62, high 186, batch 31
cpu 12 cold: low 0, high 62, batch 31
cpu 13 hot: low 62, high 186, batch 31
cpu 13 cold: low 0, high 62, batch 31
cpu 14 hot: low 62, high 186, batch 31
cpu 14 cold: low 0, high 62, batch 31
cpu 15 hot: low 62, high 186, batch 31
cpu 15 cold: low 0, high 62, batch 31
Node 0 HighMem per-cpu: empty

Free pages: 51856kB (0kB HighMem)
Active:7027353 inactive:9020404 dirty:21889 writeback:798 unstable:0 free:12963 slab:109926 mapped:3194768 pagetables:282345
Node 0 DMA free:32kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:16384kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 66287 66287
Node 0 Normal free:51816kB min:16380kB low:32760kB high:49140kB active:28109412kB inactive:36081616kB present:67878908kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
Node 0 HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0
Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
Node 0 Normal: 1162*4kB 1652*8kB 82*16kB 310*32kB 161*64kB 3*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 2*4096kB = 51816kB
Node 0 HighMem: empty
Swap cache: add 10893050, delete 10757548, find 5603628/6845075, race 0+85
Free swap = 31672172kB
Total swap = 33557992kB
Free swap: 31672172kB
16973823 pages of RAM
498539 reserved pages
32742686 pages shared
135541 pages swap cached
-----
The kernel version is 2.6

uname -a
Linux XXXXX 2.6.5-7.308-smp #1 SMP Mon Dec 10 11:36:40 UTC 2007 x86_64 x86_64 x86_64 GNU/Linux


We don't know where to start.
Could somebody help us to resolve this issue?

Regards,

C.Sacthivel

[Suman_1978]

Hi,

From the information you provided, I guess it could be an issue with memory.

I wouldnt say memory hardware is faulty but issue could be with memory is not enough or could be with SWAP memory.

http://www.linuxquestions.org/questions/linux-server-73/rhel-5-3-x86-server-freezes-803701/

Please mention what is the server model and what is memory configuration on this server.

Thanks