Informix - ADDSEG errors

[Tony Flaherty]

HPUX 11.11 patched to about 1 month ago.
Informix Dynamic Server 7.31.FD8

We are seeing the following error in the Informix log:-

hpkaioaddseg: ASYNC_ADDSEG failed, errno = 11 Resource temporarily unavailable

Informix are unhelpfully saying it's an OS problem.

I increased SEMMNU to 3072, NFILES to 114548 and max_thread_proc to 256. While this has reduced the number of occurances we are still getting occaisional errors.

The machine is a rp5430, dual processor with 1Gb RAM. SHMMAX is 512Mb, informix has a buffers value of 320Mb.

Any information and advice would be greatly appreciated.

Thanks in advance.

[Steve Lewis · Accepted solution]

/usr/include/sys/errno.h says that errno 11 is
#define EAGAIN 11 /* No more processes */

So you need to check your nproc and maxuprc kernel parameters.
Your nfiles is huge, check that against
sar -v 1 1

Given the INFORMIX error I suggest you also try increasing the value of informix environment variable
IFMX_HPKAIO_NUM_REQ
Ours is set to 3000.

Then you also need to check the number and size of virtual segments, add it all up (i.e. ipcs -ma ).

Am I right in assuming .FD8 is 64 bit IDS?

Is your o/s 64 bit too, otherwise your total shared memory allowance is 1Gb.

[Steve Lewis]

silly me - 64 bit IDS wouldn't run at all on 32 bit hp-ux. doh!

[Tony Flaherty]

Wet set the kaio limit to 2000 at the moment, what is the impact of increasing this on system resources?

nproc = 5000
maxuprc = 2048

ipcs -ma gives a total SEGSZ of 507Mb

interestingly onstat -g seg gives
Segment Summary:
id key addr size ovhd class blkused blkfree
1033 1381582849 c00000000031c000 411049984 39976 R* 50168 9
1546 1381582850 c000000018b1e000 115343360 2440 V 11626 2454
6155 1381582851 c00000001f91e000 2777088 728 M 332 7
Total: - - 529170432 - - 62126 2470

which gives a resident section of 392Mb way over the buffers value of 320Mb, I thought this was all there was in this section?

[Steve Lewis]

The increased KAIO impact on resources is minimal.

I find that a value of 2000 sometimes gives out of resource errors on a rp5400 4x650 PA8700 during heavy DB work, such as parallel dbimports and index builds.
A value 3000 hasn't given resource errors even on my superdome/xp12k.

The BUFFERS real value is higher than your onconfig value partly because of the 2.4% difference between informix and and hp-ux (just as with chunk size). As for the rest, IIRC I think the LRU control information may go in the R segment as well as the actual data but I'm prepared to be corrected on that one.

If the message you got was this:
22:03:45 hpkaioaddseg: ASYNC_ADDSEG failed, errno = 11 Resource temporarily unavailable
22:03:45 out of virtual shared memory

Then your root cause may in fact be that you have run out of swap.
swapinfo -tm will confirm or deny this, check reserved space.

Then check the IBM article on this very error message here (go to IBM IDS support, technotes/FAQs and search for IFMX_HPKAIO_NUM_REQ )

http://www-1.ibm.com/support/docview.wss?rs=630&context=SSGU8G&dc=DB520&q1=IFMX_HPKAIO_NUM_REQ&uid=swg21142285&loc=en_US&cs=utf-8〈=en

[Tony Flaherty]

Thanks for that link, it's a shame that Informix tech support couldn't find it.

I'm making some changes and rebooting tonight so we will see what happens.

Thanks for the info!

[JJ_4]

Set RESIDENT -1 in the $ONCONFIG all segments will then be forced resident.

[Tony Flaherty]

We have this set to -1.

We're having some other performance issues on this box and my colleague will be posting a new thread for these later today.

Deep joy.

Thanks all.

[Tony Flaherty]

Making the suggested changes appears to have resolved the issue.

[ckchai]

Hi Tony Flaherty,

I'm having a similiar problem here.

May I know what is the steps you did to solved the problem?

I have tried to set the IFMX_HPKAIO_NUM_REQ=2300

But still restore failed.

Thank you very much.
ck chai