Enema for at jobs?

[Pete Randall]

My at jobs seem to be stuck:

yukon(271)root# at -l
user = ops 1024552800.a Thu Jun 20 02:00:00 2002
user = ops 1024545660.a Thu Jun 20 00:01:00 2002
yukon(272)root# date
Thu Jun 20 09:37:40 EDT 2002

Why would they not have executed and how can I get them unstuck? Any clues?

Pete

[Ian Dennison_1]

The 'jobs' command should show the actual status of these according to 'at'

ie. Are they waiting in 'at' to be started, or are they running and waiting for another event? A keypress or something?

to move a job to foreground, and resume it, use the 'fg' command, with the job number as an option,..

eg fg %2 # will resume job 2

share and enjoy! Ian

[Pete Randall]

Ian,

Thanks, but jobs doesn't show anything:

yukon(273)root# jobs
yukon(274)root#

And I'm not sure how fg even relates.

Pete

[Ian Dennison_1]

How about 'at -d 1024552800.a'? what does that show?

Personally, I would trash them and start again if at all possible.

Share and Enjoy! Ian

[Pete Randall]

Ian,

At -d shows the contents of the job (as expected). These jobs normally run every night by re-at'ing themselves - except last night.

Pete

[harry d brown jr]

Pete,

is cron running?

live free or die
harry

[Frank Slootweg]

What does /var/adm/cron/log say?

It should say something like:

> CMD: 1024582129.a
> franks 3167 a Thu Jun 20 16:08:49 METDST 2002
...
< franks 3167 a Thu Jun 20 16:10:52 METDST 2002

The first two lines are the job-id, PID (3167) and the start date/time. The third line is the end date/time.

If the first two lines are present, but not the last one, then the job was started, but still busy (and hence still in the "at -l" output), and something should be visible in the ps(1) output.

[Pete Randall]

Harry,

Good question but yes, cron is running:
yukon(275)root# psg cron
root 3162 1 0 Jun 16 ? 4:26 /usr/sbin/cron

Pete

[Pete Randall]

Frank,

Excellent (I'm always amazed at how I can ignore the obvious - like checking the log). The log has this entry repeatedly:
! a queue max run limit reached Thu Jun 20 00:01:00 EDT 2002
! rescheduling at job Thu Jun 20 00:01:00 EDT 2002

What is a max run limit and how do I increase it?

Pete

[Pete Randall]

/var/adm/cron/queuedefs contains the following:
a.4j1n
b.2j2n90w
which seems to indicate that 4 jobs can execute simultaneously. I don't see how that could be the problem?

Pete

[Pete Randall]

Looking further at the log, I can see where the jobs were started but never finished (for some unknown reason):
> ops 27094 a Mon Jun 17 02:00:00 EDT 2002
< ops 27094 a Mon Jun 17 02:00:04 EDT 2002
> ops 26677 a Tue Jun 18 00:01:00 EDT 2002
< ops 26677 a Tue Jun 18 00:42:31 EDT 2002
> ops 13671 a Tue Jun 18 02:00:02 EDT 2002 - start
> ops 5172 a Wed Jun 19 00:01:00 EDT 2002
< ops 5172 a Wed Jun 19 00:41:31 EDT 2002
> ops 21535 a Wed Jun 19 02:00:02 EDT 2002 - start
> ops 1438 a Thu Jun 20 10:33:07 EDT 2002 - start

I've purged the unfinished ones and see what happens.

Pete

[Frank Slootweg]

Re: your June 20, 2002 14:42 PM GMT response:

Next time, before "purging the unfinished ones", check the *status* of the jobs. I.e., like I mentioned, the number in the log entries is the PID (process ID) of the shell that runs the job. With ps(1) you should be able to see that process and any children of that process an probably get clues as to why it/they are not finished.

[Sorry for my 'late' response. For some reason e-mail notification of new responses was not working.]

[Pete Randall]

Frank,

Thanks again, that's what I'm doing this morning - it got stuck again last night. I don't see anything in the log this morning though. I wonder if my manual trimming of the log messed up cron? If nothing else, we've got some scheduled downtime coming up this noon and I expect the re-boot will straighten things out. I'll probably have to wait til Monday to find out for sure, though.

Pete

[Pete Randall]

OK, here's the situation:

yukon(166)root# echo "/usr/bin/bdf -l" | at now +1 minute
warning: commands will be executed using /usr/bin/sh
job 1024934883.a at Mon Jun 24 12:08:03 2002
yukon(167)root# at -l
user = ops 1024984800.a Tue Jun 25 02:00:00 2002
user = root 1024934883.a Mon Jun 24 12:08:03 2002

In one minute I see the job kick off:
yukon(169)root# tail /var/adm/cron/log
> CMD: 1024934883.a
> root 26293 a Mon Jun 24 12:08:03 EDT 2002

However, I don't see the end of the job. I do a ps on the PID and all I see is the shell hanging around, doing nothing?

We used to use this technique daily and now it's suddenly stopped working.

Hmmm, I wonder if a patch affected it?

[Pete Randall]

Sorry, I meant to put the ps output in:

yukon(170)root# psg 26293
root 26293 1647 0 12:08:03 ? 0:00 sh
root 26294 26293 0 12:08:03 ? 0:00 /usr/bin/sh

[Pete Randall]

Got it !!!
PHCO_24702 introduced behavior whereby cron(1M) may no longer
execute at(1) jobs. When one of the environment variables
LC_ALL, LC_CTYPE or LANG is set when an at(1) job is
scheduled, at(1) jobs may not complete and may leave the
executing shell in a hung state.
- To work around this behavior, these three environment
variables may be unset prior to scheduling an at(1) job.
- If this work around is not acceptable, to avoid this behavior
HP recommends that PHCO_24702 be removed from all those
systems where the LC_ALL, LC_CTYPE or LANG environment
variables are set when at(1) jobs are issued. PHCO_24702
should also be removed from software depots used to installed
patches on these systems.
- The previous patch, PHCO_22767, does not exhibit this same
behavior. To ensure as many known issues as possible are
addressed, HP recommends that PHCO_22767 be installed after
PHCO_24702 is removed. If PHCO_22767 was installed prior to
PHCO_24702, it will automatically be restored when PHCO_24702
is removed and it will not need to be re-installed.


I'll try removing the patch as suggested. If that doesn't work, I'll keep on posting my musings here. If you don't see anything further from me, you can assume it worked.

Pete

[Pete Randall]

By the way, this was fixed by PHCO_27141.

Pete