Apache server hanging

[Usman]

Hi,
we have a Apache server 1.3.12 using mod_perl. the site is very busy but works normal when it is accessible from external network.

the problem is when a major part of internet can't connect to the server properly the apache seems to hang and doesn't resond. the surprising thing is that at that time apache doesn't respond even if we connect from local host. if we try to restart apache or the server it remains okay for a few seconds but then again it seems to hang from local host. probably it is due to the fact that immediately after restarting there are many requests for apache. we are also using nmap to scan local ports and at the time of hanging of apache, port 80 is shown as filtered. however during normal network conditions it is alright. the error logs don't have anything. start servers are 30, min spare servers are 20, max spare servers are 40. server uses mod_perl script to connect to oracle server however the start page is a normal script that don't connect to oracle.

any help will be appreciated,

usman

[Philip Chan_1]

Couple suggestions,

- Check if there is any system parameter overflow by "sar -v ".

- Run glance, check if your system is bounded on something. Also, zoom into the apache process, take a look at its resource usage and you may find some clue there.

Rgds,
Philip

[Bill McNAMARA_1]

is your networking connectivity still okay..
ie your networking card still active..

any chance of posting up the apache logs?

later,
bill

[Kevin Wright]

are you running a standalone server or through inetd?

[Volker Borowski]

Hi,

when it hangs, how many ESTABLISHED connections on Port 80 do you have ?

netstat -n | grep '\.80' | nl | tail

How is your swap-activity ?

This should give you an estimate, if your server config fits your needs. I think one serverprocess can handle 256 concurrent user connections. May be you should reduce the number of servers to prevent swaps ?!?!?

Do not know if this helps
Volker

[Usman]

there wasn't anything wrong with network connectivity from internal network. infact i was working on the server remotely. i also have to add that it happened only once when we were facing network bottlenecks from many parts of external networks. now it is working alright.

i did check all the system resource usage at that time. the server wasn't busier than normal. swap usage was also not significant. i also did use netstat to see the current connections. most of connection were in SYN_RECV state for sometime. i can expect that as well as considering network problems. the only thing for us to worry was that our max client setting for apache is 300 and although there was lesser connections at that time we still couldn't connect from local host. apache child should have been available to respond to our request.

[Philip Chan_1]

Hi,

Have you checked if there were any process table overflow occurred? the default process table size is 276 in HPUX 11 (nproc). My wild guess is that your system might have reached that limit by the time Apache didn't respond to incoming requests.

Rgds,
Philip