topic Re: lpd "resource temporarily unavailable" in Operating System - Linux

lpd "resource temporarily unavailable"

Anthony_141 — Thu, 09 Mar 2006 10:39:10 GMT

We are running Redhat Advanced Server 4 (kernel 2.4.9-e.62) on a ML350 G4p.

We have several G4p's loaded the same and one today stopped printing.

It had only been up about 7 days.

When you did "lpstat -o" to check the print queues we saw "Server load too high". There were no hung print jobs in the /var/spool/lpd queues.

We tried bouncing lpd, but got "dfork failed- Resource temporarily unavailable".

Top, SAR, etc showed fine. Everything else was running fine.

We ended up booting the server and lpd started normally and the users could print.

Googling did not show any useful information.

Any suggestions on what caused this and how to fix it (without booting) in the future would be appreciated.

Re: lpd "resource temporarily unavailable"

Kodjo Agbenu — Thu, 09 Mar 2006 18:34:00 GMT

Hello Anthony,

What priting system are u using ? LPRng or CUPS ?

Did you try restarting CUPS ?
Did you try to purge the printing queue ?

Try :

/etc/init.d/cupsd stop
rm -f /var/spool/cups/c00* /var/spool/cups/tmp/*
/etc/init.d/cupsd start

Good lcuk.
Kodjo

Re: lpd "resource temporarily unavailable"

Anthony_141 — Fri, 10 Mar 2006 07:50:51 GMT

We use lpd, not cupsd (hence the lpd references in my original note). But, thanks for the tip anyway!

Re: lpd "resource temporarily unavailable"

Vitaly Karasik_1 — Fri, 10 Mar 2006 08:06:03 GMT

- what is your lpd.conf?
- lpq output?
- something interesting into /var/log/messages?

Rgds,
Vitaly

Re: lpd "resource temporarily unavailable"

Anthony_141 — Fri, 10 Mar 2006 08:10:31 GMT

We never touched lpd.conf (took the defaults).

lpq and message out put was noted in original messages (Server load to high, dfork failed).

Re: lpd "resource temporarily unavailable"

Ivan Ferreira — Fri, 10 Mar 2006 08:15:34 GMT

Re: lpd "resource temporarily unavailable"

Anthony_141 — Fri, 10 Mar 2006 08:23:44 GMT

"Server load too high", dome daemons, like lpd and sendmail, won't run if the server load is high as indicated in the message, what was the server load (uptime) at the time of the problem? Some runaway process?

We ran Top, SAR, etc. and there was a normal load on the server (which was very, very low - we don't hammer the systems by any means).

Our 2 databases were running fine,and everything else appeared to be normal.

Uptime was only 8 days.

We can't rule out a "runaway process", but idle time was 95% so it's not like the system was being hogged by anything.

Re: lpd "resource temporarily unavailable"

Anthony_141 — Fri, 10 Mar 2006 11:12:00 GMT

Did just see something in the SAR logs:

Before the server was booted, %memused was at 99%. After the reboot it went down to 40%-60% used. Also it looks like we were using most of our Swap pre-reboot.

We ran TOP and it looked "OK" at the time - anyone have any other suggestions on how to trace the memory hog if this happens again?

Re: lpd "resource temporarily unavailable"

Ivan Ferreira — Fri, 10 Mar 2006 12:52:36 GMT

99% memory usage reported by sar is normal. Linux will always use all memory, if not by programs, by buffers/cache. As you rebooted the system, the buffers/cache was empty, but later will the memory. This is good.

By uptime I mean, the load average reported, not the system uptime itself, for example:

load average: 27.34, 27.20, 27.06

Re: lpd "resource temporarily unavailable"

Anthony_141 — Mon, 13 Mar 2006 09:29:35 GMT

Some load average data from right before we booted:

07:00:00 8 4137 0.01 0.10 0.11
07:10:00 5 4146 0.98 0.46 0.22
07:20:01 2 4159 0.13 0.21 0.17
07:30:00 3 4168 0.07 0.11 0.14
07:40:01 3 4176 0.02 0.07 0.09

07:40:01 runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15
07:50:00 1 4181 0.88 0.49 0.24
08:00:00 5 4201 0.05 0.22 0.24
08:10:00 5 4208 0.09 0.17 0.20
08:20:01 2 4213 0.46 0.35 0.25
Average: 5 3985 0.20 0.19 0.17

Re: lpd "resource temporarily unavailable"

Anthony_141 — Mon, 13 Mar 2006 09:53:59 GMT

Note that it should matter, but this is Redhat AS 2.1 Update 2(Redhat AS 4.0).

Re: lpd "resource temporarily unavailable"

Vitaly Karasik_1 — Mon, 13 Mar 2006 10:47:02 GMT

Anthony,

So is this RHEL2.1 (according to kernel version 2.4.9) or RHEL4?

Can you please send your lpd.conf & printcap files?

Try to add debug option to lpd.

Re: lpd "resource temporarily unavailable"

Anthony_141 — Mon, 13 Mar 2006 13:25:16 GMT

This is Redhat Advanced Server 2.1 (kernel is kernel 2.4.9-e.62)

We never touched lpd.conf (everything is commented out). Attached is our printcap file.

Also, I believe this issue started on Monday the 6th. SAR logs show swap going up to 25% used at that time and never went back down until the reboot. Gradually Swap usage grew from the 25% used to 100% used the morning of the 9th (when we started having lpd issues). Checking simlar systems normally swap is around 2-3% used.

Re: lpd "resource temporarily unavailable"

Vitaly Karasik_1 — Mon, 13 Mar 2006 15:59:05 GMT

Probably your server was really overloaded?
I suggest you to do two things:
- increase debug level for lpd ("-D" with some options)
- in addition to sar you can run "top" command via crontab in order to monitor cpu/memory utilization by each process

Re: lpd "resource temporarily unavailable"

Anthony_141 — Mon, 13 Mar 2006 16:14:18 GMT

Top and Sar looked "OK" before we rebooted (did sar -u 5 5 and we were 90% idle). Top showed nothing strange at the time.

Re: lpd "resource temporarily unavailable"

Vitaly Karasik_1 — Mon, 13 Mar 2006 16:25:58 GMT

but you spoke about unusual high memory consuming, right?

Re: lpd "resource temporarily unavailable"

Anthony_141 — Mon, 13 Mar 2006 16:32:06 GMT

SAR logs showed we were at 100% Swap usage, but running SAR from the command line showed "OK" at the time (a few minutes before we chose to reboot).

Again, we only did "sar -u 5 5". We didn't look at swap specifically from command line.

We tried bouncing lpd, ran TOP, ran SAR, checked a few other things but probably should have ran more SAR commands to get more "live" detail. We looked at the SAR logs after the reboot, which is when we saw that Swap usage started creeping up on Monday the 6th and maxed out the morning of the 9th.

I'm thinking lpd dying and not starting was a symptom, it probably wasn't specifically an lpd issue (but then again may have been). Something on the 6th started making the system use much more Swap than normal and it never went back down to a normal level until after the reboot.

Re: lpd "resource temporarily unavailable"

Vitaly Karasik_1 — Tue, 14 Mar 2006 02:29:34 GMT

>I'm thinking lpd dying and not starting >was a symptom, it probably wasn't >specifically an lpd issue

Agree!

Re: lpd "resource temporarily unavailable"

Anthony_141 — Mon, 12 Jun 2006 13:19:55 GMT

problem has not reoccurred