topic Re: program dead loop in Operating System - HP-UX

program dead loop

Son dam bi — Tue, 20 Oct 2009 01:57:19 GMT

My server was crashed yesterday , I found that the reason is because a program ( developed by our programmer ) have problem , due to the data is incorrect , the program is looping then used up all system memory , it make the system crashed , can advise is there any way that I prevent it happens again ?

ps. as there are too many programs is ruuning , I can't ask them to check all programs.

Re: program dead loop

Suraj K Sankari — Tue, 20 Oct 2009 02:10:29 GMT

HI,
>>the program is looping then used up all system memory , it make the system crashed , can advise is there any way that I prevent it happens again ?

Better you can ask your programmer to monitor there program when they test....
and before implementing they should test there program 2 3 times....
other thing you can extend your memory...

Suraj

Re: program dead loop

Hein van den Heuvel — Tue, 20 Oct 2009 03:25:54 GMT

Son dam bi>> My server was crashed yesterday , I found that the reason is because a program ( developed by our programmer ) have problem

I beg to differ. If a normal, but incorrect, program can crash a server, than that server is poorly managed. It suggests that all the knobs that are there to protect you were set to 'eleven' for that extra push over the cliff (Spinal Tap. http://www.youtube.com/watch?v=EbVKWCpNFhY )

I'm not arguing that the program is not broken, and should not be fixed. But QA testing should have stopped it production system settings may have been overly permissive without the resources to back that up.

Check out 'maxdsiz' and friends, in context of the processes allowed and swap space allocated.

You may want to check the system when it is running normally and record the various memory usages (RSS/VSS from Glance or UNIX95= ps -ef... )

>> rogram is looping then used up all system memory , it make the system crashed

If that is really true then the guilty party will be visible in the crash dump.

>> ps. as there are too many programs is ruuning , I can't ask them to check all programs.

There are likely to have been early signs and even console warning. Keep an eye on tools like TOP and Glance to see whether a process gets carried away using SYSTEM time and memory.

Suraj>> thing you can extend your memory...

au contraire, mon amie.
That's just silly. Unless you'll mean you'll have a little more time for the hurting to happen before postponed but inevitable crash.

Cheers,
Hein.

Re: program dead loop

Venkatesh BL — Tue, 20 Oct 2009 04:22:02 GMT

> the program is looping then used up all system memory

How did you confirm this? Can you show the output? I am not sure that analysis is correct.

Re: program dead loop

Dennis Handly — Tue, 20 Oct 2009 08:40:05 GMT

As Hein said, a 64 bit process can quickly use up all of swap if maxdsiz_64bit is set too high. A 32 bit process would have to work harder, unless it has a fork bomb.

>Venkatesh: How did you confirm this? Can you show the output?

Well, if there are messages about lack of swap space, that would be a clue.
Monitoring "swapinfo -tam" would indicate the problem but not the program.